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• 1 • % . PREFACE \ m 

> ' *, 

On the 10th and 11th of August 1982, the Naval Training 
Equipment Center was the host organization for a Workshop on 
Instructional Features and Instructor/Operator Station Design 
for Training Systems sponsored by the Chief of Naval Research. 
The goal of the workshop was to review current research and ( 
development work related to the .operation of the instructor 5 
station of training systems particularly with an eye for 
developing functional specifications''for these stations. 
Historically much of the research and development work has 
been sponsored by the Naval Air Systems Command and the 
Air Force"; but we -feel that many of the developments are 
applicable to a variety of simulation-based training situ- 
ations. 

Several people deserve thanks for tbeir help in making 
the workshop possible. ' CDR P. «. Curran of the, Office of 
Naval Technology cheerfully encouraged us and provided 
support, a'nd locally at the Human Factors Laboratory of the 
Naval Training Equipment Center, Catherine Bottelman^and 
Hilda Worsham both performed the many chores necessary 
during the workshop and later edited the papers and pre- 
pared the report for printing. We thank them all. 
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COMMENTS ON INSTRUCTIONAL FEATURES 

GILBERT L. RICARO 
Naval Training Equipment Center 



. < During, the past decade we have seen dra- 
matic changes/of the role instructors play in < 
device-based training and of the training < 
tasks to which simulation has"Sefin applied ' 
The ma m change we have seen has been the ex- 
tension of synthetic training to tasks which 
simulation previously could not support. Much 
of this has, been due to the development of 
software and hardware to create visual dis- 
plays, the application of new means of inter- 
acting with computer-based trainers, and the 
proliferation of smalJ computer systems with 
largfe amounts of memory.. Of course, some of 
this change has been the result of an in- / 
creased reliance on simulation for training- 
both by military training cormiands and ci- 
vilian organizations— but most has resulted 
from the greatly increased capabilities of the 
devices .themselves. All change seems to have 
us good and bad aspects, and while the ex- 
tension of simulator-based training, to complex 
tasks is most welcome, our notions of how in- ' 
Structional personnel should .interact with 
these new trainers have not kept pace with 
hardware development. Three trends have con- 
tributed to this problem: 

• 1. Device-based flight training has 
developed from the , simulations of takeoff and 
landing of a decade-ago to almost all aspects 
of the tactical use of aircraft. Largely, 
this result has come from the development of 
simulations of sensors-radar, sonar, forward, 
looking infrared, low-light level TV, and 
computer-generated visual scenes-which allow 
all members of a crew to interact in the same 
training scenario. Thus ,( individual proce- ■ 
dural training has expanded to tactical team 
training where more than one trainee's actions 
must be monitored by instructional personnel. 
As the number pf trainees participating in a 
training event increases, so also does the 
number of instructors needed. The resultinq 
requirement for the design of training equip- 
ment has been to reduce the workload and sup- 
port the communication of these instructors 
Partly this is to make their jobs easier (both 
to do and to learn), and partly this is to 
enable them to act more effectively as in- 
structors. 

2. The instructor/operator station is ■ 
the interface instructional personnel have to 
a training device and the services it pro- 
vides. The past decade has seen a good deal 
of change in these consoles. The repeater 
instruments and knobs and dials once common 
ha.ve been replaced by cathode ray tubes, alpha- 
numeric and multifunction keyboards, joy- 
sticks, and the like. Today training tasks 
are selected by entering data in response to 



y choices presented in menu form on programmable 

* displays and light pens are used to select 
many of the ootions. On the horizon, we see 
flat panels with touch sensitive surfaces re- 
placing- much of the input/output hardware in 
current use. Such systems present in succes- 

* sjve displays the information and choices that 
previously were spread over console surfaces t 
These systems pose the desiqn problem of re- ■ 
casting the rules of the spatial qrouping of - 
isCormation and choices into successive tem- 
poral groupings. In the future we expect that 

a single surface will serve to present infor- 
mation and record inputs, and system desiqners 
shall have: to be. both Selective and efficient 
m their choice of what to display at a qiven 
time. and what information to require from in- 
structors. Rules, or at least guidelines, 
will have to be developed for moving about in 
menu-driven displays, for recovering from 
incorrect inputs, and for keeping track of 
choices that have been made.. This problem of , 
engineering the flow of events which .present ' 
information to instructors and allow them to 
y exert control over a device seems to be at the 

heart of deficiencies in today's complex ' ' 
m trainees. In future systems, probably no- 
where will the conflicting requirements of 
allowing for flexible usag'e and of providing 
i for standardised instruction need to be more 
delicately b^flan^ed than at the operator's 
interface to the -device. 

3. The past decade has seen great in- 
creases of the processing power and storage 
capability of computer systems-all at ever 
falling prices. It has become common to see 
^ training devices witji several minicomputers 
networked to process an appropriately parsed 
problem, and we expect that the future will 
only see 4 more of this as microcomputers are 
devoted to smaller and smaller elements of the 
information processing and display tasks 
This is true now for many of the calculations 
involved in a simulation and will be true also 
for the services a cjeFice offers its users. * 
From an instructional point of view, these 
developments allow possibilities previously 
considered impracticable— where the computer 
system of a trainer not only supports the 
simulation of a training task but also stores 
and manipulates data'useful for instructional 
control. Today's computers are not only data 
processors, but machines to store and operate ' 
upon data bases as well as engines that can 
manipulate symbolic relations and make infer- 
ences. These new capabilities, emerging as 
software with supporting hardware, will allow 
training systems undreamed of a decade ago. 
The problem for design then is to define how 
such services should operate and how to fit 



them into operational training devices. 

% These three trends have resulted in the 
requirement to define better the instructional 
role a training device's computer capability 
should fulfill and to define functionally the 

-services a training device should support. 
Research and development efforts have stressed 
the observation that devices should provide 
not only a simulation of a task byt should 
also be equipment to support the training, of 
that task. Much of t ( he research work has ex- 
amined instructional features that, have been 
incorporated into current devices or has sug- 
gested new features. Presently instructional 
features fall into two classes: those A de- 

. signed to support real-time instruction using 
a device and those designed to provide off- 
line services for instructional or administra- 
tive personnel. Many of the real-time train- 
ing features are old friends such as the abil- 
ity to freeze a task, set environmental con- 
ditions, move vehicles and the like. Re- 
search efforts have produced others such as 
models to determine the actions of targets or 
the output of voice synthesizers* computer 
measurement of a trainee's progress-, and 
schemes to automate feedback for training. 
Features designed for off-line support tend to 
be relatively new because* the storage required 
for them has only recently become availab/e. 
Here we have seen developments in the keeping 
of students' records, in providing briefing 
and debriefing information, in the incorpora- 
tion of software to tutor instructors on the 
use of a device, and in programs to define and 
create new training exercises. While neither 
of these lists is inclusive, they do provide a 
flavor of the developments we expect to see- in 
the training device of the future. 

E very* so often it is prudent to review the 
ways we do things and - the techn/iogy we employ 
to do them. From problems ^jncfvered in train- 
ers currently operational, it\is clear that 
now is the time to examine the- functional de- 
sign of the instructor/operator stations of 
training equipment. We need to be able to 
take advantage of hew technology by knowing 
how to design it well so that it can support 
training .easily. The goal of this workshop 
then is to provide information for the* devel- 
opment of new systems as well as to present an 
overview of the knowledge currently available. 

• Large and growing literature exists on 
many of the topics related to the design of 
instructor/operator stations. These include 
topics such as instructional systems devel- 
opment, workplace design, and the utilization 
of training devices as well as discussions oT 
the design of man-computer dialogues and de- 
scriptions of display and control equipment for 
man-machine interfaces. Only a small portion 
of this writing is devoted to the design and 
operation of training equipment,' and for "the 
most part it exists as a technical literature 



which may not be widely distributed. Because 
of this, a bibliography fo.llows that lists 
many of the reports discussing instructional 
features and console operation particularly 
for training appl ications. 
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INSTRUCTOR/OPERATOR STATION RESEARCH AND DEVELOPMENT AT AFHRL/OT 

Dr. Harold D. Warner 
University of Dayton Research Jnstltute 



INTRODUCTION 

The Operations Training Division of 
the A1r Force Human Resources Laboratory 
(AFHRL/OT) has as Its mission the Improve- 
ment of A1r Force combat effectiveness 
through training related "science and tech- 
nology* Located at VfUiams Air Force 
Base, Arizona > AFHRL/OT performs this 
mission through ^ two primary functions: 
behavioral RID ta solve flying training 
probl ems through improved technol ogy ; and , 
engineering R4D to develop training devices 
N as vehicles for training R4D. The person- 
nel who support these two areas form a 
diverse, mul tidisclplinary team of 
specialists ranging from research psycholo- 
gists, research Instructor pilots, and 
human factors "specialists to aerospace 
engineers, mathematicians, and computer 
programmers • AFHRL/OT operates and -ma i n- ' 
tains two of the nation's most advanced 
simulation facilities for training and R&D, 
the ^Advanced Simulator for Pilot Training 
'Asm at Williams Air 'Force Base, and the 
Simulator for Air-^-Air Combat (SAAC) -at 
Luke Air Force 'Base. At present, the ASPT 
1s configured with an F-16 and an A-10 
cockpit, and the SAAC provides two F-4 
cockpits for simulated air-to-air combat. 

AFHfcL/OT R&D efforts have been accom- 
plished in a variety of areas " in^uding 
simulator training effectiveness, training 
methods, pi^ot performance measurement, and 
pi^ot perception and cognition. In the 
area of simulator training effectiveness, 
efforts have been directed toward the 
development and assessment of visual 



display 



terns and 



display generation 



techn 1 ques^^WicJ udi ng he Imet-mourrted vi sual 
displays and computer generated imagery; 
the evaluation of alternative motion cueing 
systems such as the G-seat, G-suU, and 
cockpit motion platform; and, the deter- 
mination of visua 1 display* requirements 
including f ie^d-of-view, co^or, resolution, 
aeria 1 perspective, and terrain cues. As 
for the area of training methods, efforts 
, have addressed, for example, whole vs. 
part-task training using full-mission 
flight simulators and desk-top. special- 
function trainers. The area of pi^ot 
performance measurement has al s6 been 
strongly supported and includes experiments 
on meastfres of basic flying and combat 
skills, pVot workload, and stress. In the 
area of pfldt perception and cognition, a 
'variety of pllbt^factors have been examined 



such as decision making, tactical planning, 
and visual cue utilization. 

Because the efficacy of aircrew training 
is dependent not only on the training capa- 
bilities of flight simulators but also on 
the quality of flight Instruction, AFHRL/OT 
has applied considerable resources to the 
design and* evaluation of instructor/ 
operator station (IOS) controls, displays, 
and workstation layouts as well as the 
evaluation of various IOS configurations. 
The specific efforts that were accomplished 
.by AFHRL/OT; and th? activities in 
progress, are as follows. 

AFHRL/OT IOS RSO ACCOMPLISHMENTS 

!./ MO Flight Simulator IPS / An IOS 
was designed for the Air Force A-1(J Opera- 
tional Flight Trainer/ This effort involved 
/a determination of the IOS controls and 
displays required for instructing simqlated 
A-10"flfght and . the design of an A-10 IOS 
consistent with these requirements. In 
this R4D effort, a groQp of A-10 instructor 
pilots {IPs) waS observed during the con- 
duct of flight simulator training in the 
Advanced Simulator for Pilot Training. (ASPT) 
and each IP was administered a questionnaire 
when the training was concluded.. The 
questionnaire was designed to assess what 
controls and displays were used to instruct 
the student pilots and what should be added, 
to the IOS to facilitate flight instruction. 
Both the observational and questionnaire 
data were analyzed to determine the control 
and display requirements for an A-10 simu- 
lator IOS* ♦An IOS "was subsequently designed 
which incorporated these controls and dis- 
plays. In general, the IOS consists of 
/seven Cathode Ray Tubes (CRTs), A-10 
repeater instruments^ a variety of controls 
a CRT hardcopy unit, 
video monitors which 
display the studeh£ 
visual scene, and 
Circuit television 



and indicators, and 
Two of' the CRTs are 
would be used to 
pilot's out-the-window 
one CRT 1s a closed 



monitor that would enable the MP to monitor 
the 1h-cockpit • activities of the studer\t' 
pilot.. The remaining CRTs would provide 
informational feedback about * the students 
flight performance. A variety of displays 
were developed fd** these CRTs and the 
function and operation of each display were 
determined. The IOS also included A-10 
repeater instruments which were arranged* 
the sa<ne as they are in the actual aircraft. 
Additional ly> a number of controls were 



EMC 



incorporated in the IOS for the executive 
/"~^ and^administrative controV of the training^ 
process. .The specific configuration * ggt ' 
— — t these \ controls was provided and the 
function, operation, and layout of the 
control were specified. This IOS design 
effort was fully described by Gray, Chun, 
k Warner, and Eubanks 

?. A-10 Training Feat ures Eval u ation . 
The utility of various frisIructionaT fea- 
tures of the ASPT IOS were evaluated in the 
context of A-10 fight training, Six of 
the features available on the A§PT IOS were 
, evaluated. ' They v/ere: problem and para- 
meter freeze, rapid initialization, auto- 
matic demonstration, automated performance 
feedback, self -confrontation, 9 and task . 
difficulty. In the study, IPs were required 
to rate the utility of each instructional 
^ feature and a systematic observational 
procedure was used to determine the fre' 
quency with which each feature was used by 
the /IRs. The data analysis indicated that* 
performance feedback and initialization 
/were 4 rated highest and were used most 
frequently. Problem and parameter freeze 
was a 1 so used regularly. Many of the 
features, such as automate* - performance 
feedback, task difficulty, preprogramed 
demonstrations, and sel f-confrontation, 
were rarely used,. Some of -the features, 
such as freeze and initialization, were not 
used in an instructional manner, but more 
for training management purposes. That is, % 
they tendetf' to be used to terminate a 
•training exercise and to provide a transi- 
tion to the next exercise, rather -}han 'for 
providing opportunities to give the student 
pilots performance feedback and the 
-appropriate remediation. This study is 
documented in the report by Gray, Chun, ^ 
- Warner, and Eubanks M981). 

3. F-15 Simulator IOS Operational Test # 
x and Evaluation . An operational test ancT 

evaluation or an F-15 flight trainer, IOS 
was conducted. Measureii)ents of the 10$ 
controls and displays, workstation dimen- 
sions, and workplace environment were 
compared with the published ~ military 
Standards for equipment design. These* 
measurements induded*CRT display character 
size, .pushbutton control size, lighting 
levels, ambient temperature, noise levels; 
•writing surface depth- and height, and reach 
distances. In addition, a number of F-15 
IOS users were interviewed to determine 
operational deficiencies and recommenda- 
tions were provided for correcting these 
deficiencies." This evaluation was accom- 
plished in 1977 for the Air Force Tactical 
Air Warfare Command 'TAWC). 

4. F-5E Instrument FMght Simulat or 
Opera tionaTHest and Evaluation. AFHRL/ot 



supported an operational test and evaluation 
of the F-5E Instrument Flight Simulator. 
* This effort involved the development of the 

Vtest procedures and data collection 
materials as well as assistance in the 
collection '-and analysis of the test data. 
The data collection, materials consisted' of . 
rating forms which were administered to 
F-5E IPs to determine the fidel i ty and 
trainingT capability of the simulator 
cockpit and the operabil ity and instruc- 
tional capability of the IOS. The data 9 
collection effort involved: (1) briefing 
the pilots and IPs on the missions to be 
performed; (2) distributing the* rating , 
forms; (3) monitoring the activities of the v 
IPs at the IOS and 4 the pilots in the 

, cockpit; and (4) recording their comments 
concerning the simul ator. The evaluation 
was managed by the Air Force Test and 
Evaluation Center (AFTEC). 

' 5. A-10 Operational Flight Trainer 
Operational Test "and Eval uation"! The 
operational test and evaluation of^the A-10 
Operational 'Flight Trainer was also 
supported by AFHRL/OT. The support involved 
assistance iiO/the development of data 
collection materials and collection of test 
c data. The data collection materials 
* consisted of rating forms that were admin- 
istered to- ,the test IPs to assess the 
fidelity ^and- training capabilities of the 
A-10 simulator cockpit iffd the instruc- 
tional capability of the A-10 IOS- This 
, evaluation was also managed by AFTEC. 

6. SAAC IOS 'Control Panel Design- The 
contfol /indicator panel x for "the Aeri al 
Combat Engagement Di spl ay ( ACEO ) of thfe 
Simulator for Air-to-Air Oombat (SAAC) was 
redesigned. The ACED i-s essentially a CRT f 
which is capable of displaying several 
pages. These pages present a view of the 
gaming" are^ and the various aiYcraft in the 
area, a pilot's view from inside the cock- 
pit of the visual scene, and performance 
scoring data* Controls a)id indicators are 
used to di spl ay di f f erent pages and^ to 
cftwsge range scales. In the modified panel 
design, the ACEP control s/indicators that 
, were original ly contained in two separate 
panels were* consolidated into a single # 
panel . The control s/indicators *£re 
arranged ^in functional groups. The panel 
design has been implemented on the SAAC* m 

AFHRL/OT IOS R&D IN PROGRESS 

1. IOS Design Guide . A guide for 
flight simulator IOS design is currently 
under development by AFHRL/OT. The, guide 
will be made up of three* sections, or 
, volumes. The first volume will include 
detailed descriptions of advanced flight 
simulator IOS configurations. ^ The second 
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* will, contain a compilation of the appro- 
priate human .engineering design criteria. 
The' third will pKOvide an IOS design for 
fighter/attack 'flight simulators based on 
the data in the first two vo 1 times. At 
- present, several simulation' training 
facilities hive been^ visited and descrip-; 
tions of the IOSs have been prepared. * 
The facilities that were visited include: 
H) F-l 5 Instrument Flight Simulator- (2) 
A-1P Operational Flight Trainer; (3) F-14 
Weapons System Trainer: (4) Tactical Air- 
crew" Combat Training System /TACTS)' and 
(5) EA-6B 'Weapons System Trainer. During 
these visits -the operation of several other, 
simulation systems was observed, namely/ 
the F-14 Operational Flight Trainer, F-14 
Mission Trainer, TA-7J Operational FMght 
Trainer, A-6E Weapons %stem Trainer, and 
A-6E Night Carrier Landing Trainer. Tactics 
information will not -be included 'in the 
report because of its classified nature. 
Descriptions of the IOS for the F-16, F-18, 
and F-4 trainers' will also be incor- 
porated in the report. The sepond volume 
will* contain relevant human engineering 
design criteria, standards, and specifica- 
tions obtained from a compilation of 
existing data and from studies conducted at 
AFHRL/OT.^ Examples of the areas that will 
'be addressed in the r IOS design guide 
include control /cii spl ay design and layout, 1 
workstation conf i guration , workpl ace 
environment, anthropometry, and operator 
seating. Vo 1 ume three will provide a 
conceptual^ IOS design 'specifically for 
fighter/attack simulator appl ications. * It 
win contain detailed design specifications 
and the function and operatic* of the IOS 
control and displays will be explained. 
When the three-volume report is completed, 
.a sequel will be prepared^ for tanker/ 
transport/bomber simulators/ „ 

2. IPS Interactive r CRT Controls 
Evaluation . R5D has > been initiated, in- 
which alternative control devices for IOS 
interactive . CRT displays will be 
evaluated. The controls are: (1) touch 
panel, (2) light pen, (3) numeric keyboard, 
and (4) voice actuation. These controls 
will be compared in relation to three 
different CRT presented performance tasks. 
In one task, the subjects will be asked to 
load various weapops from a menu onto the 
pylons of a stylized aircraft. The proce- 
dure is to select an appropriate pylorf 
number with the controls, then se.leqt and 
enter the weapon number identifier. For 
the second t^sk , the subjects wi 1 1 be 
required to move a cursor from a specific 
location to another location using the 
controls, which is ' analogous to initial- 
izing a simulated aircraft at a new loca- 
tion on a CRT, presented IOS navigation 
map. The third task _will require the 



subjects to enter numeric data to establish 
the simulator parameters suc(x as altitude, 
airspeed, heading, radio frequencies, 
latitude, longitude, and glide slope. Ttje 
subject sample wiT1 # consist -of. IPs who will 
be tested in a repeated measures experi- 
mental design. Bafch response times and 
errors will be recorded to provide the 
, measures of task performance. To date, the 
CRT displays (performance tasks), are nearly 
complete and the' controls are operational. 
t The test/will be conducted on a special- 
function i part-task trainer which is com- 
prised of a Chromatics video display 
'terminal and a North Star minicomputer. 

3. # ASPT IOS Energy Haneuverabil ity 
Di spl ays7 Energy Maneuverabi 1 i ty Di spl ays 
(EMOs) were developed for the IPS of the 
Advanced Simulator for Pilot Training 
(ASPT), one for use in conjunction with 
j F-16 flight training and the other for A-1P 
. training. These displays are presented on 
an IPS vector graphics CRT, and they show 
'>£he current energy state of the simulated 
a) rc raft . Si nee ai rcraf t maneuverabi 1 i ty 
is a function of available energy, these 
d* spl ays wjl 1. enabl e the simul ator IP to 
monitor the maneuvering capability of the 
^ aircraft. With this information, the IPs 
can instruct their student pilots on how to 
increase energy . to maximize aircraft 
performance without exceeding the struc- 
tural limits of the aircraft. An experi- 
ment will be conducted to assess the 
uti 1 i ty of these di spl ays for f 1 i ght 
instruction at the ASPT IPS. A group of 
F-16 and A-1P IPs will be asked to monitor 
the corresponding F-16 or A-1P EMD during a 
series of preprogrammed ,fl ight maneuvers in 
the. ASPT. Following the demonstration, the 
IPs, will be asked to rate, the instructional 
utility of the EMD and thei'r cortments' w1l 1 4 
be solicited via a questionnaire. The EMDS 
implemented in the proposed study were, 
adapted from the EMDs used on* the Navy's'. 
Tactical Aircrew Combat Training System 
(TACTS) which were .original ly developed by 
Pruitt (1979). - * 

4. IPS Tactics Display Bevelgprnent , 
RiD ha*s been Initiated to develop ar^JOS 
— display for a1r-to-surffcce tacffctt , 
training. A corfceptual model of the display . 
has been designed which will be implemented 
and evaluated on the ASPT. The display 
provi des the essenti al ai rcraf t f 1 ight 
parameters and tactical informatfon such as* 
altitude, air*Sp§ed, and G-load as well as 
an overhead view of the tactical area with 
fixed range rfngs to show the range and 
bearing of ground radar and surface-to-air 
missile launches. The simulated aircraft 
is positioned' in thecenter of the display # 
and the terrain moves in accordance with 
the speed and heading of the aircraft. In 
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the evaluation, a group of IPs will* be asked 
to monitor the display- during a series of 
preprogramed tactical missions 1n which 
radar will lock .on and missiles win be 
fired/ The IPs will be surveyed to deter- 
•mine' the utility of the display for 
instructing/monitoring pilots in simulated 
tactical training scenarios. / t , • 

5. IPS* Integrated Visual Pjsplay 
De velopment^ An experimental effort Ts 
underway — to deve'op an IOS Integrated 
visual display for u5e by IPs during basic 
(T-37) simulator flight training. The 
purpose *of the display is to provide all 
the critical flight instrument data a 
single CRT. similar 1n form to a head-up 
display 'HU&1. In this effort, a question- 
naire was dev^oped and a<4n1nistered to 25 
T-37 IPs to determine the display require- 
ments for Instructing/ monitoring basic 
training missions including contact flight, 
aerobatics, formation flight, Instrument 
flight, and navigation. The questionnaire 
data are currently being analyzed to 
identify the specific flight information 
required for each of the missions. The 
appropriate display symbo^gy wiT be 
selected to depict this information -and a 
display wl 1 ! be designed containing the 

symbo^ogy. . 

* * • 

6. IOS Instructional Features Re 3 u J • 
ments< A - survey was recently initiated to • 
TTIenTify the utiHty of advanced instruc- 
tional features in aircrew training devices. 
The survey wil 1 be administered to IPs at 
various flight simu 1 ation facilities. The 
IPs will 1>e asked to rate bow frequently * 
the IOS advanced instructional features are 
used in flight training, how easy is it to 
use each feature, and * the 1 training value 

.*and"tra1n1ng potential of the features. A 
' total of 17 , features^ will be examined 
including Freeze }{ Record/Playback, Reset, 
demonstration, Crash Override, Automated 
Performance feedback, 'and Automated Adap- 
tive Training. -The survey will be conducted . 
at A-10, F-15, and F-4 -simulator training 
.sites. The results of the' survey will, be 
used as a gdiide for specifying the required 
Instructional features Ijn futile flight 
simulator IOS designs. 



CONCLUSIONS 

■ ■ ■■ * 

AFHRL/OT 'has provided scientific and 
engineering support to a variety of flight 
simulator IOS design 'and £vali*a£1on efforts* 
Many of these efforts have Seen in.sunport 
of outside Air Force agencies which Tiave 
involved thr assessment of IOS instructional 
utility, and the development of generalized 

.guidelines for IOS design. Other efforts 
have been accomplished strictly in house to 
solve specific IOS design problems... It is 
anticipated that AFHRL/OT will provide 
continued, if not ' expanded, IOS R*0 
support- Greater emphasis will be placed 
on the use of desk-top/ special functions 

' trainers in the conduct of J future * IOS 
design studies. These devices will- be used 
to Investigate JOS control and display 
designs as building blocks for the develop- 
. ment of full -mission flight trainers. 
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OPERATIONAl/PROBLEMS IN INSTRUCTOR. OPERATOR STATION " DfiSIGNS 
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- INTRODUCTION , * • , 

In the past, training devices, as' 
with many other systfems, were all too * 
often "provided". to the. user and left 
for "him to use as best he could. The 
* user neither -had been involved in thfe 
.specification-^ior in the desigh of the 
device. , As a result, the operator in- 
terface or instructor operating sta- 
tion ' (IOS) .was generally designed as a' 
simulation control unit rather than as 
a training control unit. Fortunately, ' 
the early devices. simulated relatively 
^simple .systems and w'^re selddm used 
for ifull system or crew training. - 
4 Thus," the instructor with the help of 
-a technician, could generally ''oper- 
ate" the device. " ; 

Two factors have significantly . 
changed this picVure. First, the 
application of advanced digital tech- 
nology to training devices has expan- 
ded their capabilities enormously. 
Existing weapon system trainers (WST) 
can provide multiply targets "in ah 
electronic and missile warfare en- 
vironment with visual and motion simu- 
lation. Freeze, dynamic replay, re- 
set and demonstration functions are " 
cfenerally available^. Graphics dis- 
plays provide the instructor with 
"pilot view" display* and plots* ^Hun- 
dreds of pages of alphanumeric ydata 
on simulation optipns are provided. 

The second factor was the in-* . 
creased emphasis on the use of simu- 
lation for training which occurred in 
the 1970s. Rising costs of weapon 
system ownership and operation led to 
a renewed application of simulation * 
to crew training.' 

Thus, with the requirement for ex- 
panded use of simulation and with the 
technology available to implement the 
requirement, a wide variety of train- 
ers were developed and installed. 
Each major weapon system training site 
now has at least a.W£fT, and in addi- 
tion, most have part task trainers - 
(PTT) to support crew position train- 
ing. 

Unfortunately, research and de- ^* 
velopment effort in 105 design did -not 



follow £te p>ace set by) simulation 
technology. Relatively simply con- 
soles designed* primarily to control, 
the simulation* had proven acceptable 
'for the early devices, even though 
not optimum, in th£ absence of any 
more refined requirements, - the .con- 
sole^ for the *iew sophisticated train- 
ers have followed much the same "Sp- 
proach. Unfortunately , the conse- 
quences have produced serious ope^a- 
bility problems as* well as sub-opti- 
mum training support.. .While ^the 'de- 
vices carv^come close to duplicating 
the weapon system characteristics, 
•they cannot necessarily be Operated 
to train effectively using that capa- 
bility. 

The Naval Training Equipment 
-Center undertook ?l series of studies 
in the 1970s to look at instructor 
console problems'. One of these stud- 
ies (Charles, 1975) looked at the role 
of the instructor pilot (IP) in simu- 
lator training. The study which sur- 
veyed all of the aviation readiness 
training squadrons and the related- 
simulator installations fbund'that 
several/changes had occurred in the 
opera^ipn and use of the devices. <of 
-mdre/iijfportance was the expressed * 
copfeerh that the next generation of 
trainers which were., then in design- . 
and dewlopment , would prove inoper- 
able by instructor personnel if the.- 
simulation control orientation of the 
IOS design continued. The* extensive 
use of cathode raf tube (CRT) dis- 
plays would compound the problem. 

IOS PROBLEM AREAS ' ' 

C • 

When operability problems- did 
arise in some of the newer WSTs, the 
NAVTRAEQUIPCEN -undertook a review of - 
the IOSs. with the goal of identifying 
bofh the causal factors and feasible 
solutions. Two- WST IOSs have been re- 
viewed, th£ EA-6B WST Device 2F119 < * 
and the F-14A WST Device 2F112...In 
general, the problems which were iden- 
tified, are similar in both devices. ' 
They ref lect^ twQ basic development, 
problems, the lack of -human favors' 
engineering effort during design ahd 
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development and the 'failure to define 
the operational environment *and re- " , 
quirement. for the device in sufficient 

detail* * • . 

*• # -» 

While the human engineering pro-i 
blems are significant and interact ^ 
with tfie operational problems tp com- 
pound the operafi&LW-ty prpbleri, the 
operational problems are of prime im- , 
portfenae.^ The best human engineered 
IOS cannot solye the user or dper- % , 
ational problems which exist. However 
conversely/* the most user responsive, 
system will also be. ineffective "if in- 
operable. 

Although steps have been and are 
being -taken tp solve some of' the oper r 
a*bility problems, the training <?r; uti- 
lization problems in general rpmain. 
These problems are centered about - 
three areas: > 



a. definition. of the u§er 
b". definition of the use 



c. . definition of the training m 
v v . fun^ions . «" . 

d. ' definition of the operators" 

• While the thxtee are clearly in- 
terrelated, each has a unique impact 
on' IOS design Requirements 'since they 
identify the singulation- requirements, . 
the trainer 1 and training functional 
.requirements and the operator/instruc- 
/tor T *interface requirements,,, 

^USJBR DEFINITION . . \ 

The user- refers to the -units^- 
which actually utilize the trainer in 
their training program. In the.^s. 
Navy/ the user and -the custodian land ^ 
maintpiner are not t^e ■ same ■ activity. 
Most of the WSTs are supported by the 
Fleet A^iTation Specialized Operational 
Training .Group* (FASOTRAGRU detachments. 
The 'trainers are maintained and read- 
ied, for training as scheduled by 
FASOTRAGRU persbnriel. They ^Iso pro- 
vided technical support to the Oper- . 
ation oi the.* devtpe • The primary 
< Risers are the MteeJ: Readiness Sqifa- 
. drons (FRS) and the fleet Squadrons, 
' Their 'requirements, although similar 
in terxte^of the weapon system and *ts 
operationT^iff^r Aft- terms of the 
training program content and X^lemenr 
tation* 



• ' The FRSs copduct several levels 
of. a syllabi designed to familiarize 
the replacement aircrew' with'£he 
weapon system and its operational 
missions. The majority pf replace-, 
ment aircrew ate recent graduate^ o'f v 
•the Naval Air Training Command and , 
'are designated Naval Aviators and 
Naval-Flight Officers. They ca«/be t 
considered as the bachelors degree of 
the aviatiofr fcommunity^and have the 
basic professional qualifications. 
The FRS training* programs are de- 
signed to produce; aircrews Vho are 
qualified to pperate a specific * * 
'weapon system and to join a fleet*- 
squadron for additional. training prior 
tp deploying. Although tony - of 'the- . 
training programs have been t task ana- 
lyzed and behavioral an£. training ob- 
jectives ' identified, the thrust of 
the typical FRSrtraining prograrp is 
.not criterion-based, nor is training 
to proficiency the. primary objective. . 
The FRS has been likened 'to. a 'graduate 
school . As in academic graduate • 
' schools, .while grades are utilized, 
they are considered more for informa- 
tion-. th$n for a. pass-fail functioac 
The replacement aircrew are qualified 
aviators and NFOs 7 andkthey are con-^ 
. sidered to be capable ~f transitioning 
to the new system in the allott.ed , 
training time/ When ; unsatisfactory 
performance does occur, -it is initial- 
ly handled with the usual options of 
C9unse-lmg, extra time and tutoring. 

In ^limnjary, the FRS training prp- 
gram,/given- the limited training time, 
limited flight time, and limited * simu- 
lator 4ime, ;is organized .to give the . 
replacement aircrew the maximum train- 
ing and experience possible while 
cicely monitoring performance l for 
safety and correct operating 'proce- 
dures^- " 

,Fleet squadron training is* simi- 
larly conceived and except for NATOPS 
and instirumfent checks which have 
associated performance criteria, the 
rest 'of the training is aimed at im- 
parting as. much experience and know- 
ledge across the mission requirements 
as possibly prior |to deployment. 
Readiness criteria 'have been developed 
but are, in general, more experience 
oriented rather than performance* ori- 
ented ♦ 4 

Theae training programs depart 
from.-the classic training approach 
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#hich is organized around a highly 
structured syllabus, testing 'and fixed* 
Criteria. <u 

/ ". * * 

• In suittmary, the discussion of the 
User programs, has emphasized*' the in-, 
strucfcor oriented nature of the pro- 
gram since, it serves to frame some of 
the operational problems in existing 
IOS design. 

UffE DEFINITION 

«■ 

The^FRS use of the WST varies as 
a' function o'S the phase of training. 
The FRS syllabus ^begins with systems^ 
familiarisation and progresses through 
basic flight characteristics to* weapon 
system mission 'and tactics *feraining« * 
Procedures' and* position training aret.*6 
conducted on**PTTs if' they are, avail- ^ 
able. If not, the WST is usfed to 
support all phases of training includ- 



variables involved. Some instructor 
interaction .is, however, recjuire'd to 
reorganize, or reorient the problem ^ 
based on aircrew actions. 



\ 



• system familiarisation 

• basic flight . 

• bperation and basic tactics 

• aglvanceyd 'tactics ^\ 

** * 

• special events 

-$ 1 " 

Procedures and basic flight train- 
ing, when conducted on* "the WST; gen- 
erally utilize a manual training mode 
in which the instructor can control 
the initial conditions and the evolu- 
tion of the subsequent event s-v- The v 
instructors consider it essential to 
control the onset. and removal of mal- 
functions and emergencies,* They also 
find it easier and more realistic to 
simulate the various controller func- 
- tiqnk in a manual- mode. 

- Systems operations and basic 
•tafctics training is generally conduc- 
ted i*7r a semi-manual mode if possible. 
Pre-prog^anune<3 initial condition sets 
are used for establishing the target (s) 
and environment as well as the flight 
starting-point. The instructor* then 
assumes control and adjusts the seer 
n^rio to meet aircrew performance and • 
training needs. - «* ^ 

Advanced .tactics* and battle 
problem training events are more "fully, 
programmed because of the almost im- 
possible task of trying to control * 
manually the targets and the many 



^ Special events such as. spin, high 
angle of attack and missilfe defense 
training are ^generally conducted in 
the manual mode since they involve 
"free f light" the student,' " 

Finally, alt^hpugh not yet imple- 
mented, programmed' and standard 
mission events are needed to permit 
,crew and unit evaluations. SiSSiila- 
tion provides a -feasible* means of 
accomplishing such operations sihee 
realistic targets and weapons (and 
weapons effects) are generally un- 
available for "shoot-of f s" . 

Fleet squadron use is similar to 
FRS^use except that familiarization 
training (procedures and basic flight) 
is not part of the training program. 
Detailed training event guides and 
grade sheets are generally not utili- 
zed. 

In summary, the use of the WSTs 
in 4 .FRS and fleet squadron training 
includes : 

• manu&l modes for familiariza- 
tion training (if conducted .on the 
device) as well as for special "ex- 
periential" events such as spins and 
missile defense. 

• semi-manual modes for systems 
and basic ^tactics training 

• programmed or 'formulated 
modes for advanced tactics and battle 
or war problems 

TRAINING FUNCTIONS DEFINITION 

Effective training requires a 
series of tasks ranging from instruc- 
tor preparation ' for the event to re- 
cord keeping. The set of tasks used 
to' analyze the IOS designs included 
the following: . . 

a. Prepare - review event, air- . 
' crew files, simulator status; 

get forms, guides/ manuals 

b. Brief - review event, objec- 
tives and procedures with 
aircrew and instructor staff 
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c. , Initialize - configure con- 

soles and cockpit; select .and 
implement initial conditions 
or programmed raision 

d. Train - instruct; control sim- 
ulation; monitor performance 

e. Evaluate - evaluate aircrew 
pr6£iciency; diagnose perform- 
ance/ problems 

* j 

f< De v brief -Review event results 
with aircrew and instructors 

ST. Manage data - update aircrew, 
staff y(S simulator files 

h. Develop- events - create/pro- 
. . gram/modify events, displays, 
missions, simulation data 



i. Train instructor - train in 
simulator operation and use . 

The training device should pro- 
vide support to each of the funotions, 
especially the brief, initialize, 
train, evaluate, debrief and develop 
functions. * . 

t 

DEFINITION OF OPERATORSf 

The traditional operators of 
training devices have consisted of the 
. technician simulator operator (SO) 
provided' by the FASOTRAGRU Detachment, 
and the ^instructors provided by the 
squadrons. The SO is typically a 
technician who is learning console - 
operation prior to being assigned to 
a maintenance crew. Thus, in general, 
the SO is not an expert in simulator 
operation although such assistance is 
generally available on-call. The SO 
is available to assist the instruc- 
tors in^ ''button smashing"*. 

The instructors from th£ FRS will 
have completed the instructor training 
sy fW bus which generally includes a 
short course on simulator ^use and 'some 
oh-the- job. experience. Of more im- 
portance is *he fact that the instruc- 
tor will also be instructing on other 
trainers as veil as in-flight. There- 
fore, wit'h rare exception, he is not 
dedicated to JfST instructing. During 
his tour of duty, he wili>, also be 
assigned other squadron duties and his 
ground instructing time will taper off 
as a function of time fend ^duties in 
the scjuadron. 

The fleet squadron instructors 



are notUrained in simulator operation 
(except for any prior FRS experience) 
-and cannot be expected to become qual- 
ified m operating the trainer. 

SUMMARY ' 

The user data which should con- 
strain^the JOS design involves the 
specification of the:* 

- t 

a. user - primarily FRS and 
fleet squadrons 



b. 



use - primarily system oper- 
ation and tactics (except 
when supporting familiariza- 
tion training) 

functions - all training 
tasks 



d. operator - the technician 

operator 4nd the squadron in- 
structors. The operator is 
generally new to the device, 
the FRS instructor has lim- " 
ited training in operating 
the device, and the fleet * 
squadron instructor will I 
have no training in operating 
the device. 

"OPERATIONAL PROBLEMS^ 
i 

r A variety of problems occur in 
operating typical WStfs, many of which 
seriously impact training effective- 

. ness. Some of these problems, which 
result from failure to consider the 
operational environment in terms of 
the user, the use, the training func- 
tion support and the pperators are 
outlined below. Jhe problems wilf be 
grouped in terms of los configuration 
and layout problems, display and con- 
trol problems, operating problems and 

-training function problems. 

IPS LAYOUT PROBLEMS 

» 

From the. review of IOS install- 
ations, it is clear that insufficient 
attention is given to console location 
and layout, in general, it. appeared' 
thdt the typical IOS is located in a 
main traffic flow pattern, and be- 
comes a meeting place for 1 personnel 
in the area. While observation of on-* 
going training can contribute to 
learning, already room environment is 
not conducive to either the training 
or to beneficial observation. Figure 
1 depicts the location and layout of* 
the 2P119 IOS. All traffic to and' 
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from the cockpit must pass 'through the 
area. In addition to the congestion 
problem, the aircrew can generally get 
a glimpse of the upcoming simulation 
being initialized. Light spills onto > 
the displays when doors are opened. 
Noise interferes with inter-instructor 
communications . 

ARRANGEMENT PROBLEMS 

Most IOS consoles provide a sta- 
tion for each of the instructors as 
.foreseen by the specification and~he 
design. This rarely reflects the use 
and user needs. One of the major 
problems occurs in .the battle or war 
^ problem event, especially for the 
fleet squadrons. As can be seen in 
Figure 1, only three instructor sta- 
tions exist (and none for the SOi) 
The battle probl.em instructor who is 
responsible for the pverall evolution 
of the event and for evaluation £nd 
critiquing of crew performance, has no 
• effective station from which to oper- 
ate. The IP' mans the flight station; 
the ECMO instructor, the tactics sta- 
tion, In practice, the battle problem 
instructor mans the ECMO 1 station 
where he is forced to share the dis- 
plays of. the other stations, manipu- 
late the station control "between the 
flight and tactics modes of operation, 
.and perform the ECMO 1 instructor 
T functions. The problems are similar 
for the 2F112 a& can be seen in Figure 
2. Here, the battle problem instruc- 
tor is forced to sit behind the in- , 
structor stations and 'function with a, 
clipboard and whatever display he can r 
read from this position- 

DISPLAY PROBLEMS 



. The CRT display" proyidedv the de- 
signers the opportunity for 'displaying 
literally any simulation data avail- 1 
able. In rnariy of the WSTs, the oppor- 
tunity was utilized. The- volume of 
. patfes* required the 'use of index pages. 
- Table 1 summarizes the display options 
available on the 2F119 WST flight sta- 
* tion displays. 

- ' "Ther quantity of data far exceeds 
the capacity of any 1p to access- and 
.utilize effectively during training 
and still be able to monitor and eval-. 
uate replacement pilot performance. 
The. options cannot be "a'ccessed by an 
instructor who is not trained and pro- 
ficient ot experienced in the flight 
station operation. A similar set of i 
displays exists at the tactics station. 




instructor Area 



Figure 1. Device 2F119 IOS Arrangement 



.TABLE 1. 2F119 FLIGHT DISPLAY OPTIONS 



Title 



Display Index 
Initial Conditions Index 
Initial Conditions 
Pilot Instrument Monitor ^ 
Pilot Console Monitor 
ECMO, 1 Monitor 
Procedures Monitor Index 
Procedures Monitor 
Malfunctions Index 
Malfunctions 
Imput Codes 
Function 

Parameter Recording 
Cross Country 
Hostile Environment 
Terminal Area 
ACLS 
GCA/CCA 
CEM Index 
CEM (Alphanumeric) 
CEM (Graphics) 
CEM Summary 
Demo Index 
Demos 
DRED 

Visual Status Monitor 
Graphics Test Display 
Memory Monitor 
System Time 

TOTAL 



Pages 



1 
1 
10 
1 
1 
1 
2 
99 
1 
13 
2 
10 
1 
1 
1 
1 
1 
1 
1 

variable 
1 
1 

. 1 
10 
1 
3 
'2 
3 
5 
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OPERATOR 
STATION , 



Figure 2. Device 2F112 IDS Arrangement 



A similar display approach was im- 
plemented on the 2F112. 

* CRT cockpit monitor displays^ on 
the typical console pose serious prob- ■ 
lems since the data is not displayed 
in >a manner readily interpreted by. the 
instructor who is also training in 
flight. The displays do npt typically 
parallel either the arrangement or the 
format usjed in the aircraft. Figure 3 / 
depicts a typical page used, in WSTs 
arid is from, the 2F119. As /can be seen 
the instructor cannot glanfce at the 
display and ascertain switch settings 
without first locating th£ control by 
reading the list and then reading the 
control setting. 



CONTROL PROBLEMS 



/ 



The typical instructor station 
has two CRT displays along with some * 
cockpit repeater displays. } .The CRTs 
are operated by selecting ,£he CRT (and 



sometimes the specific area on the CRT) 
to be utilized, then selecting the 
'display mode to be accessed and fi- 
nally pagina to the data required. 
The sequence of steps which involves 
both switch and light pen operations, * 
for example, is time consuming and 
requires the instructors concentrated 
attentions. Errors, which often occur, 
typically require repeating the entire 
sequence of steps. 

Light pens are widely used for 
simulator control. Although light 
pens are becoming more reliable, they 
are still unacceptable for time sen- 
sitive inputs such as malfunction in- 
sertion, weapons launches and training 
control functions. iA the past, poor 
light pen reliability has severely 
handicapped console operations. 

The typical joystick display- 
control dynamics involves a step func- 
tion which renders it unusable for CRT 
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operations. ManuaJ control or flying 
of targets using the joysticks has 
proven equally impossible beqause of 
the lack of sufficient flight infor- 
mation and because of, joystick control 
axes coupling. 

Cockpit* configuration and initial 
condition control mismatches generally 
require extensive manipulation 6f IC 
procedures to resGlve the problem. 



Manual modes of trainer operation 
have generally proven difficult* to 
"utilize even though required for many 
events. The exception is the IC which 
involves Starting with the cockpit pre- 
f light procedure £t the take-off end 
of the runway. However, even for this 
IC, tfae instructor, unless he accesses 
reinvent data £ages is unsure as to 
fuel state, stores configuration, 
weather, etc. The major problem is 
one of knowing on what page and in 
which display mode ttfe relevent data 
is available, and when accessed,' how 
to edit and what the impact will be on 
inter-related parameters. The task is 
beyond the "novice" user. 

Programming or formulating meth- 
ods requires extensive training and 
recent experience to utilize. This 
requirement cannot be met by the 
typical instructor. The procedures 
on most WSTs do not provide an "inter- 
face"" between the /instructor and the 
mission programmer. 

Target creation is normally ac- 
complished in simulation parameters 
rather then in user terminology. Thus, 
targets are created in terms of "small 
fighter" with an "IR" missile and a 
spot jammer, for example, rather than 
in terms of a Badger or Backfire or 
•Muffc<Db. The result is that the in- 
structor does not generally know what 
the displayed target represents or 
how to evaluate ^the aircrews tactics 
relative to the target. 

TRAINING FUNCTION PROBLEMS 

Ideally, a training device should 
be atrle to support each of the train- 
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ing tasks ihvolved in the training 
mission. E*ew do. To the extent that 
they do" not, ^additional" workload is 
placed on the* instructor or SO and 
additional training may be required, 
-fable 2 is a summary of three WSTs 
support £o the 'training functions out 
lined earlier. 



TABLE 2. 'TRAINING FUNCTION SUPPORT 



Function 


2^111 


WST 
l 2F112 


2F119 


Prepare- 


none 


none 


none 


Brief V 


none 


none 


none ■ 


Initialize 


yes ' 


yes 


yes 


"Train — 


yes 


yes 


yes 


Evaluate 


none 


none 


some 


Debrief 


none 


none 


none 


Manage Data 


none 


'none 1 


none 


Develop events 


yes 


yes v 


. yes 


Train instruc. 


none 
r -v 


noije 


.none 



As can be seen, the typical WST-s 
are not designed to support training 
functions. As has been discussed in 
the previous sections, even tho^e 
tasks in Table, 2, which are recorded 
*as "yes", present marginal support to 
the function. Yet, each of the de- 
vices has data stored within the sys- 
tem which would be useful for the in- 
structor in: 

• reviewing the scenario ^nd 
options prior to briefing 
the student 

• reviewing the weapon system 
operating procedures and simu 
lation options. 

*• • briefing the aircrew on event 
procedures , scenario and 
objectives * 

• briefing the instructor staff 
on the scenario and training 
procedures 

• debriefing the aircrew on the 
results of the training 

• -debriefing the instructors on 
the problems and changes re- 
quired 

• updating training records 

Communications simulation is 
performed manually in most trainer . 
and can require up to two instructor 
full time for some battle problems. 



.Communication options controls on 
most WST IOSs have proven so' time con- 
suming to use and so error producing 
that the options are not utilized. 
The control panel is left in "over- 
ride". 

OPERATING PROBLEMS 



Simulation, of controllers and provid- 
ing relevent background communications, 
especially for sequential battle prob- 
lems/ is almost impossible. 

None of the WSTs reviewed pro- 
vide the capability of either briefing 
or debriefing the aircrew (or instruc- 
* tor staff) without utilizing training 
time and the instructor console. None 
of the WSTs provide the additional 
displays and controls required or the 
interface required. 

J 

% , Hard copy output on most WSTs is 
typically so slow and involved that it 
.is not used. Yet, all instructors 
- agree, that hard copy is desirable, 
especially if no debriefing displays 
are available. 

Malfunction options far exceed a 
usable set on most WSTs (typically- iw 
the hundreds). In addition, -informa- 
.* tion on the simulation characteristics 
of the malfunction is not provided. 
As a result, the instructor has no in- 
formation on the different cockpit in- 
dications and relevant procedures, for 
example, for different engine fires of 
flight control system failures. 

SUMMARY 

Recent reviews of the IOSs of 
several WSTs have, documented a variety 
of operational problems. In, general, 
they result f roin both the failure to 
< • consider ueer requirements or to de- 
sign the IOS ta basip human factors 
engineering criteria. 
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ABSTRACT 

The real world is defined as the operational training environment. 
Instructional support features (ISF) are simulator hardware and software ^ , 
capabilities that allow instructors to manipulate,\upplement and, otherwise 
control the student's learning experiences with the \ntent of promoting the 
rate at which skills are learned and maximizing the Revels of skills achieved. 
This paper summarizes research findings, observ^tions'Xand 'experiences aealing 
with the design and use of the following ISFs: automac^J demonstrations; 
record and replay; programmable and manual malfunction control; automated 
cueing and coaching; automated controllers; computer controlled adversaries; 
and quantitative performance measurement. Definitions of tn&s&atures are 
given, followed by comments on their design an/J use. A final seb^jon deals 
with persistent needs for improved .instructor training in the t$e oK§imula- 
tion in general and in ISFs in particular. ^ 



BACKGROUND 

The increasing complexity^)? modern 
weapon systems, combined with their ever 
increasing performance capabilities, have 
created a situation in which the performance 
* capabilities of crewmembers are becoming com- 
,mensurately more critical. As crew perfor- 
mance capabilities become more critical, the 
training of those capabilities becomes an 
increasingly difficult problem in that' less 
performance variance can be tolerated. 

Training usingfoperatidfra^ equipment is 
becoming increasingly difficult* *Acqu1si-_ 
tion, operation and maintenance costs are all 
high* Skilled maintenance and instructional 
personnel often are scarce. Finally, envi- 
ronmental constraints constantly encroach 
upon real world training areas and impact on 
how they can be us*ed. ' / 

The use of aircrew training devices 
(ATDs), while not as glamorous as inflight 
training, has several inherent advantages. 
ATDs still are relatively inexpensive to 
procurer, operate and maintain, at least in 
comparison with actual equipment. Conse- 
quences of Operator error or inability to * 
appropriately respond to normal and emergency 
requirements are significantly less in a sim- 
ulator. Control over training events is 
vastly superior to that available in the air. 
Simulators can be used to create low proba- 
bility conditions at will for training pur- 
poses, training events easily can be repeated 
until necessary operator ski IT levels are 
achieved, and their Utilization rates are 
significantly greater than actual equipment. 



The question row has, become how to max- 
imize the training utility of ATDs rather 
than whether to use them in training. With 
the proper design, application and use oT 
instructional support features (ISFs) in 
training devices, the utility and advantages 
of training devices over actual equipment 
can become even greater. The training sim- 
ulator can become an even more integral part 
of the real-world operational training envi- 
ronment. 



INSTRUCTIONAL SUPPORT FEATURES 

ISFs are features of training simula- 
tors that are specifically designed to facil- 
itate the instructional process. ISFs are 
the hardware and software capabilities that 
allow instructors to manipulate, supplement 
and otherwise control the student's learning 
experiences with the intent of promoting the 
rate at which skills are learned and maxi- 
mizing the levels of skills achieved. They • 
are designed to allow control of instruction- 
ally related variables such as rate Qf con- 
tent presentation, amount of content pre- 
sented in a single block, amount and distri- 
bution of practice, types of tasks practiced^ 
knowledge of results, and measures of perform a 
mance. ISFs are the capabilities that trans- * 
form a simulator from simply a practice de- 
vice into a flexible element of the total 
training system. Common instructional sup- 
port features include freeze, record and 
replay, and programmable initializing con- 
ditions. 
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Early ISF implementation efforts were 
frequently viewed as compromising the hard 
won fidelity of simulators. While it cer- 
tainly is possible to implement ISFs in ways e 
that will compromise the fidelity of simula- 
tion, lower levels of fidelity are not the 
necessary consequence of introducing ISFs 
into a training simulator. The features can 
be 'implemented in ways that will leave the 
fidelity of the simulator irvtact and at the 
same time enhance the training effectiveness 
and utility of the device. The commonly 
assumed justification for high levels of 
fidelity is to enhance training effective- 
ness. Instructional support features are 
capable of still further enhancing training 
effectiveness and efficiency. Thus, instruc- 
tional support features can be regarded as 
"fidelity plus." This statement assumes 
that the implementation of the support fea- 
tures is accomplished in a professional 
manner that does not compromise the fidelity 
of the simulation itself (e.g., video dis- 
play terminals used for instructional feed- 
back are not placed in middle of an otherwise- 
accurate instrument panel). 

This paper does not attempt a discussion 
of each ISF available with current technolo- 
gy. Rather, several were selected W«fche 
basis of difficulties that seem to persist in 
their design and use. The content offlhis 
paper is based in part upon the ratheVTHBfc^ 

»vh Hata that d^tit Om 




Automated demdBffNti ons ; 
Record and repfay; 
Programmable "and manual malfunction 
control; * 

Automated cueing and coaching; 
Automated controllers; 
* ' Computer controlled adversaries; and 
Quantitative performance measurement. 

Definitions of the features are given, 
followed by comments on their design and use. 
A final section deals with persistent needs 
for improved instructor training in the use 
of simulation in general and ISFs in partic- 
ular. 

Automated Demonstrations 

Automated demonstrations (auto-demo) 
permit the standardized presentation of a 
mission segment or entire simulated flight. 
All simulator systems, including the visual 
scene (if present), motion system, primary 
flight controls and displays, crew communi- - 
cations, and sensor displays respond as 
though skilled aircrews were at the controls. 



Auto-demo requires substantial two-way 
communication between ATD controlling soft- 
ware and hardware, and crew station controls 
and- displays. Given this requirement, 
auto-demo is found only in more sophisticat- 
ed computer controlled ATDs. Technically, 
airto-demo can be constructed for any segment 
of flight (e.g.,*takeoff, fly straight and 
level, perform aerobatic or air combat ma- 
neuvers-, deliver weapons, fly a standard 
approach, and land) without the student or 
instructor operating any primary controls. 

Two instructional values commonly are 
assumed for auto-demo. First, the feature 
provides a performance model that the student 
can observe, analyze, pattern his own be- 
havior pattern after, and use as a reference 
for self-evaluation in subsequent trials. 
The student's workload is considerably re- 
duced during auto-demo, providing him with 
a better opportunity to observe relationships 
among cues and system responses. Similarly, 
instructor workload is considerably reduced, - 
allowing a better opportunity for instruc- 
tional interaction with the student. 

A second value is that automated demon- 
strations may be the only way to show the 
student what is expected of him in ATDs that 
exactly reproduce crewstation physical con- 
figuration. When the crewstation is either 
single seat (e.g ; , F-15) or incorporates 
controls and displays necessary to execute 
manual demonstrations at only one crew posi- 
tion (e.g., F-4), there is no opportunity 
for an instructor to enter the crewstation 
and do a hands-on demonstration. Therefore, 5 
the only remaining avenue for a demonstra- 
tion is by means of auto-demo. 



to-demo utili- 
ullivan (1981) 
lue of an auto- 
greatest when 
performance 
ted are new to 
not highly 



In an examination of au 
zation, Semple, Cotton and S 
concluded; "The training va 
mated demonstration will be 
the cues, responses and task 
requirements being demons tra 
the student (i.e. , they are 
familiar to him)." - 

Navy research with the A-7 Night Carrier 
Landing Trainer (NCLT) has shown that auto- 
-demo has a "significant" instructional value 
(Brictson and Burger, 1976). However, there 
were no data quantifying the specific con- 
tribution that the feature made to overall 
training effectiveness or efficiency. This 
application does represent a case, however, 
where an instructor-flown demonstration was 
not possible because the ATD is a single 
seat configuration. No other method for 
demonstration was possible. 

The oaly known experimental study to 
address the auto-demonstration capability 
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was reported by Hughes, Hannan and Jones 
(1979)* This study attempted to address the 
instructional impact of both the auto-demo 
and record and replay instructional features. 
Subjects were divided into three groups with 
each group receiving the same basic instruc- 
tion followed by either extra practice, re- 
corded replays in lieu of one practice trial 
in each block of trials, or an auto-demo in 
lieu of one practice trial in each block of 
trials. The results indicated that extra 
practice was most instructional^ beneficial, 
followed by record and replay 'and autp-derao 
in that order. The generalizability of these 
conclusions must be questioned, however, in 
that the basic instruction for all three 
groups incorporated auto-demo, and automated- 
instruction was used in lieu of instructors. 
Given a real-world instructional setting, 
these results might nof hold-up. 

There are two basic methods for creating 
auto-demos. One is. to use an ATD's record 
- and replay capability A This method requires 
that a proficient amrew member fly the 
necessary profiles in the desired manner 
using desired techniques. This method has 
the advantage that highly profi-cient crew- 
members, usually are available and the dis- 
advantage that it is difficult, even for , , 
highly proficient aircrew members', to fly 
profiles with the perfection often judged 
necessary for a performance model for the 
student. Also, the lack of well defined 
criteria for the largest percentage of fly- 
ing tasks.makes it difficult for instructors 
to agree qn what constitutes an acceptable 
demonstration. 

The second method for creating an 
auto-demo is to develop customized software 
for each demonstration. This approach has 
the advantage that mathematically perfect 
demonstrations can be created. It has the 
distinct disadvantage, however, of requir- 
ing computer software specialists' to create 
new demonstrations or to modify initial 
demonstrations. Both manufacturer and* u&r 
personnel view the direct software approach 
as the more time consuming and costly of the 
two methods available. 

The need for "perfect" demonstrations 
must be questioned for either method of 
development. Demonstration of "typical" 
performance may be more appropriate in that 
no one flies "perfect" maneuvers in the real 
,world. 'Appropriate tactics, techniques and 
perhaps even typical errors and performance 
problems should be emphasized, rather than 
prjecise flight path control. > 

Record and Replay . - ' 

Record and replay is the capability of 
simulators to record relevant system . 



parameters and then use these data to re- 
create student's performance. Typically, the 
last five .continuous minutes of performance 
are recorded. After freezing the ATD, the 
instructor selects the point in time during 
the last five consecutive minutes that he 
wants replay to be initiated. All eVents of 
consequence are reproduced during replay, 
including the visual scene (if present), 
motion systgm operation, primary flight con- 
trol and display movements, and appropriate 
sensor displafy content. The ATD performs as 
though the student was again flying the re- 
played segment exactly as it had been flown 
before. 

« • 

Experience has shown that record and 
replay is used more often, in the training of 
tasks that are new to the student and are 
relatively complex (for him). In general, 
the use of record and replay has centered on 
undergraduate pilot training and on the , 
training of new (to the pilot) and relatively 
complex advanced flying, skills, including 
night carrier landing and air combat maneu- 
vering. 

The primary instructional benefit at- 
tributed to record and replay is that it 
provides both students and instructors with 
an objective recreation of the student's per- 
formance that can be examined for problems, 
errors and their causes. In short, the 
student is provided with objective knowledge 
of results and the instructor with concrete 
evidence from which to suggest areas or, 
techniques for the improvement of the stu- 
dent's performance. Rej^y seems to be most 
useful when'the student ; *fs not aware that he 
had made an error or is uncertain of the pre- 
cise cause of his ^rror and resulting per- 
formance problem. This situation.jLsvmost 
apt to arise -when complex tasks arebaing, 
initially learned. Following interviews 
with instructor pilots throughout the mili- '< 
tary, Semple-eft al. (1981) concluded:' "The 
training value of record and replay will be 
greatest when the cues", responses and task 
performance requirements being learned are*: 
new to the stud^t*" 

1 Record and replay is currently used 
almost exclusively for pilot and/or copilot 
training. Replay could be used, however, in 
a^crew training environment as a meaffs to 
evaluate and improve crew interaction. 
Voice replay also would be* required in this 
application since some of the better indica- 
tors of crew interaction are crew communica- 
tions. This potential value for replayi-de- 
pends heavily on how well individual crew- 
member responsibilities are defined. 

f- 

The record and replay feature requires 
substantial two-way communication between 
ATD controlling softftare and hardware, and 
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crew station displays *nd controls. Thus, 
the feature js found only in more sophisti-^ 
cated, computer controlled ATDs. ATD hard- 
ware and software must be configured to in.- . - 
duce changes in the operation pf simulated 
subsystems without intervention by the stu- 
dent or .the instructor. 

Several factors should be considered 
when implementing a record and replay fea- 
ture. For replay, a total of five minu*s , 
appears to be-more than adequate. In most 
uses, 1.5 to 2.0 minutes probably would be 
sufficient. The amount of replay time to 
be incorporated into an ATD shouldbe deter- 
mined on a case by case basis. Ar^lterna- ; 
tive to specific time increments also should . 
be given consideration. A reasonable alter- 
native would be to provide the instructor 
with a cueing control. Replay then could 
•be started at. \ determined time prior to 
when the cueing control was activated (e.g., 
20 seconds; or whatever would be reasonable 
for the particular training application). 
ThTs would allow instructors to relate re- 
play to training events rather than clock 
time Voice replay should receive serious 
consideration for training tasks where voice # . 
communication is a significant element of 
-total task performance. Finally, there may 
be training and user acceptance values in 
being able to deactivate replay part way 
through, thereby allowing the student to 
assume manual control using a -fly out 
capability. * 

Malfunction Control 

/ Procedures trainers, part task trainers," 
operational flight trainers and full mission 

. simulators generally include capabilities to 
simulate a variety of malfunctions. It is 
widely accepted that ATOs provide a safe and t 
controlled environment- for training re- 
sponses to malfunctions and emergencies. In 
such settings, responses to single and mul- 
tiple malfunctions can be trained and 'prac- 
ticed either in isolation or in mission con- 
texts. The number of .malfunctions that in- 
structors can present tor a student typically . 

• ranges from 60 to several hundred. The num- 
-be>r typically used ranges from 20 to bU. 

The most common,method of inserting and 
removing simulated malfunctions is manual 
operation of controls by the instructor. 
Types of controls now in use include dedi- 
cated pushbuttons, programmable pushbuttons, 
alphanumeric keyboards, and, touch panels. 

v Another alternative is automated mal- 
function insertion. One assumed value of 
automatic malfunction insertion and removal 
is that it unburdens the instructor from 
more routine tasks , freeing his-attention 
and time for more important instructional 



activities. A second value Involves" student 
N self practice. Some training managements 
encourage students to practice whenever ATD 
' / time is.available, even if instructors can- 
not be present, In these cases, automatic 
malfunction insertion could be used to struc- 
ture training sessions. 

Little creative thought has been applied 
to meaningful 'methods for automatically in- 
- serting malfunctions. Time into-the mis- 
si on is the most common method and has proved 
unworkable because mission time often does 
not correlate, with mission events. For 

• example,' ATD clock time into a mission typ- 
ically does not take freeze into account. 
If the ATD is frozen so the instructor can 
work with the student, the malfunction clock . 
keeps running. As a* result, a. new malfunc- 
tion can be inserted at a very inappropriate 
time in the mission.. The acceptability of 

/ this circumstance is low, both instruction- 
ally and in "terms of 'user acceptance. 

Semple, Vreuls, Cotton," Durfee, Hooks 
and Butler (1979) developed a functional^ 

• specification for a simulator instructor s 
console which, among other things, incor- 
porated automated malfunction insertion and 
removal.^ In the initial concept, the in- 
structor could select a malfunction and 
select from a list of initiating conditions 
for that malfunction. The malfunction would 
be inserted when the initiating conditions 
were met and would be removed following the 
completion of correct student responses. In 
art experimental prototype of the system, 
automatic malfunction insertion and removal 
were incorporated, but instructors did not 
have a choice of initiating conditions. 
During a preliminary test of the system, 
instructors did not respond favorably to the 
automated malfunction scheme (Semple, 
Their strongest complaint was lack of flex- 
ibility. However, instructor training in 
system utilization was minimal, ,and it is 
not known whether their comments would hold 
if instructor training in the feature had 
beeri more rigorous. 

. 1 There are four general issues which 
'should be considered when implementing mal- 
function insertion options in an ATD. One 
is the amount of malfunction cue recognition 
' which should be trained in versus out of the 
ATD Using valuable training devices to 
train content, which would be equally appro- 

• priatei for other media, is inefficient train 
ing de'sign. Ease and convenience of use by 
instructors an y operators is equally impor- 
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If a feature is difficult or trouble- 
n use, experience shows that ,it is 
.to go unused (Semple, et al.; 1981). 
aids sftouTd be designed into the 
, to remind instructors Which malfunc- 
are engaged *and v^'hich are available. 

J 



Finally, .with automated malfunction inser- 
tion, programming must.aJ.low for meaningful 
failures relative ta the' mission rather than 
to an -arbitrary time-line and instructors^ 
must be able to override automated insertions 
and removals.. ( * «v 

Automated Queing and Coaching 

Cueing messages alert students, 
e.g. "check altitude.'" Coaching mes- 
sages instruct, e.g. ">£tttm to 
flight level 150." 'Automated cueing and 
" J coaching are not common in present ATDs. 
Their assumed instructional values center 
on th& promgtijess and accuracy of guidance 
and feedback information provided to the ^ 
student, and the unburdening of instructors 
through automation. Such systems also £an- 
provide feedback and guidance to students 
when a qualified instructor may not be pres- 
ent, as in "extra time" practice. They 
require good automated performance measure- 
ment to determine what messages should be, 
given to the student and the timing of their 
delivery. Automated cueing and coaching sys- 
tems can be disruptive if messages occur too 
frequently, which suggests the need to be 
able to deactivate the system, and further . 
sQggests the desirability of. decision logics 
designed to keep the frequency of cueing and 
coaching messages witiiin acceptable bounds in 
relation to student skill levels. 

A programmed mission scenario typically 
i-s required, so that desired performance is 
defined clearly. A quantitative performance 
.measurement (QPf-f) system and additional, de- 
cision logics also are required for deter- 
mining message content and timing. *A QPM 
capability is needed to sense when student 
performance is less than what is required for 
the task he is practicing. When differences f 
are found, system logics are needed' to /iden- 
tify the appropriate message content. Typ- 
ically, a cueing message would be transmitted 
first and performance monitoring would 2 con- 
tinue. If the performance deficiency was 
not corrected, the appropriate coaching mes- 
sage would be transmitted. If the deficiency 
continued, either coaching ntessages could be 
continued or the instructor alerted so that 
he could intervene. 

*Three" technologies pre available for 
creating the messages to be transmitted tb 
the student: audio tapes; digitally stored ' % 
speech; and computer generated speech. Com- 
puter generated speech is relatively new but 
is readily understandable. ; /Digitally stored 
speeciVMs even m6re""hurrran." , Taped messages 
often involve prolonged search times and 
mechanical unreliabilities. Digitally stored 
or computer generated speech. technologies are 
well suited to automated. cueing and coaching 
message delivery because, the messages- 



involved usually are brietf* and computer 
memory requirements are well within reason. . 
Also, changes in message content are easily 
- accomplished. 

The issue of which messages should be 

* built into an automated cueing and coraching 
system must be addressed on a case by case 
basis. The analysis should be'gin by identi- 
fying typical performance problems that go' 
unattended by the student. Commonalities 
among the performance* problems (and associa- 
tion cueing and coaching messages) then 
should be identified* so that the smallest 
possible set of cueing and coaching messages 
can be identified. Draft messages then 

_ should be developed and reviewed for clarity 
and brevity. Finally, the automated cueing 
and coaching 'system should be tried out in 
a representative target operational training 
setting before its design is finalized. 

Present applications of automated 
cueing and coaching center on basic flight 
and navigation task training. In the future, 
it may be pdssib.le' tq'incorpdrate these capa- 
bilities into procedures' task training. In 
one application, use of , the feature varied 
considerably from instructor to instructor, 
which was to be expected .because of lack r 
of instructor training on potential values 
and limits.. Sonie ^instructors indicated that 
the feature was used "quite often" by stu- 
dents who came in to practice on their own, 
but they could not quantify the amount of 
f use. * , 

It is likely that the use of looser per- 
formance tolerances is desirable early in 
training to trigger cueing or coaching mes- 
sages. This cou\d se?ve to reduce the num- 
ber of messages transmitted to the student * 
during early training whep hi§ abilities to 
^ perf orm^may be significantly less than at the 
** conclusion of training, and when distractions 
may1)e of negative training value. However, 
there presently are no guidelines for deter- 
mining 'tliese tolerances. Further research 
is needed. * v 

Automated Controllers ' ^ 

'r " * , 

Control means to regulate on direct. 
Instructors often play the roles of air > 
traffic controllers, tactical controllers, 
or they control the actions o*f simulated air- 

* borne threats. The automated controller 
instructional support feature is designed to 
assist ATD instructors irT providing the con-^ 
troll er function. 

Automated controller systems incorporate 
models of the specific operational situations 
that they control and require automated per- 
formance measurement capabilities to relate 
actual student performance to desired 



performance. .Controller messages are then 
appropriate to„both the original incoming 
message and the operational situation. The 
combinationiof automated speech understand- 
ing, situation Recognition and computer gen- 
erated speech are becoming powerful 'instruc- 
tional tools for automated controller appre- 
ciations, A typical example involving both 
speech understanding and speech synthesis 
might be: 

\ 

Incoming communication: * 
. From Pilot. 

"Approach Control - X RAY^l 
turning to final" 

Situation: 

~ % Aircraft X RAY 1 is turning 
, onto the final ILS approach 
'at the correct altitude. 
Environmental conditions are 
those selected for the exercise. 

Outgoing Communication: 

From Automated Controller ( 

"X RAY 1 you are cleared to 
land. Wind now 150 at 20 
gusting 27" 

Synthetic \toice-based controllers are 
expected to be used increasingly for many 
ATO voice applications with highly struc- 
tured vocabularies. 

Potential training benefits steming 
from automated controllers lie jn four areas: 
1) unburdening instructors and/or ATD oper- 
ators-from playing controller roles; 2) in- 
creasing the timeliness and correctness of 
controller messages and feedback; 3) unbur- * 
dening instructors from the^measurement of 
verbal task performance (and associated 
record keeping); and 4) providing a -new 
medium thraugh which students and ATDs can > 
interact in a highly natural planner. As 
examples of the fourth pointy it is tech-, 
nicaVly possible for thevStudent to ask the 
ATO, "How did I do on that bomtj run?" If 
the system has the necessary performance^ 
models and performance measurement capabili- 
ties, it could respond: "Very well," and 
provide a detailed performance diagnosis if ( 
desired. Automated voice technology also ^ t ' 
opens opportunities for very, precise, auto- 
mated studenj-co^ching and cueing. 

Current training ^ystgnis which incor- . 
porate' computer speech^ understanding are ' 

• limited ^individual word recognition (1'IWR) 
technology to* interpret individual words pr 
Very short phrases, This technology re- 
quires very precise, stylized*speech by the 

* human and requires each student to repeat 



each phrase or word up to,lQ times to -"traiir" 
the computer to understand what was said. 
In a recent prototype -training system evalua- 
tion, recognition rates 50 to 97% were found 
with an average of 85% (McCauley and Semple, 
1980). This is far below the 95 to 99% ' v 
recognition rates possible under i deal con- * 
ditions (Lea, 1980). Connected speech recog- 
nition technology has recently surfaced, al- 
lowing peoplfe to speak more naturally, with- 
out the £tylization constraints required, by 
IWR*. Also, connected speech systems seem 
to be easier to "train." 

With respect to computer generated 
speech, present technology is quite a4equate 
for creating words and sentences that can be 
understood by the human. Work continues on 
ways to make the computer generated speech 
sound more natural. Finally, much of the 
• technology needed to create the mathematical 
models and performance assessment "capabili- 
ties- required by automated controllers also 
exists. However, it still is the' case that 
all such models require experimental testing 
and fine tuning. 

The design of automated controller 
models involves two principal considerations 
First, the technology of computer speech 
understanding is improving very rapidly. ^ 
Computer speech technology'developments in 
the last, Seven years have emphasized "appli- 
cation^ 1 rather than the development of 
basic principles of speech understanding and 
synthesis. This has resulted in certain sys- 
tem inadequacies at this time, but dramatic 
x improvements are currently under development 
and will become available irT.the near future 
(Cotton and McCauley, 1982). 

- A secondhand important consideration is 
the design of the operational performance 
model th'at drives the controller. Early 
controller models, for example, y/ere derived 
" from "text book" procedures for, performing 
the maneuvers that -were being controlled. 
In the operational world, pilots seldom fly 
profiles strictly according "to procedures. 
Humar) fontrollers are aware of this and re- 
spond accordingly. For example, a pilot may 
Choose" to turn to intercept 4iis final ap- 
proach course. at a distance from touchdown 
and'at an altitude that an automated con- v 
troll er has not been -programmed to recognize 
as the initial point for a final* approach to 
{ landirjg. Two things can result. 0)ie is 
that the controller model may issue spurious 
conmands because it has not Correctly recog- 
nized initial conditions for' the start of 
the approach. Secorfd, the automated per- 
formance measurement system that provides 
information to the controller model alsamay 
.be "fooled" because of a departure from the 
procedure it has been prdgrammed to accept 
as baseline performance. This can. provide 



/the controller model with inaccurate* infor- 
mation and can result in a poorer store on 

• the approach than actually -was* earned. 

* ' An automated. ground controlled approach 
(GCA) controller has been successfully ap- 
plied to an F-4E simulation. as an integral 

m part of the Automatic, Flight Training Sys- 
tem (AFTS) (Swink, 'Smith* Butler, Futas and 
Langford, 1975), This instructional appli- 
cation of automated speech technology marked 
the beginning pf.a new era"' of jwtdfoated con- 
trollers* for ATDs / While the F-4E AFTS GCA 
controller required a resident general pur- 
pose computer and a disk memofy. systertrto 
provide a limited repertoire of GCA oriented 
words and phrases, modern microelectronics 
technology now offers similar capability on 
2 to 4 chips with rqpertbires of up to 200 
words. The AFTS technology was-, however, 
found^sufffcient for it's purpose and 'sub- 
sequently employed in A7 Air National Guard 
and Gr^k Air Force training. ' 

•» » 

There are many prototype systems either 
under development or in testing which incor- 
' porate automated controllers including the 
'Navy's F-14A operational flight trainer 
(SempJe et al., 1979), the ftrecision 
Approach Radar Training "System (PARTS) \ 
(McCauley and Semple, 1980) and an Air 
Intercept Controller (AIC) tuning system 
(Mctauley, Root and Muckl er" 1««2) . 

The use of controller modfHfc in air- 
screw training is relatively ney% The tech- 
nology and the "lessons leamed^ata bases 
require expansion. It is strongly ^recom- 
mended, therefore, that all automated con- 
troller jnodels be evaluated and refined 
•during the development process .to ehs.ure 
that the models function accurately before 
their design is frozen. 

'Computer Controlled Adversaries 

Computer controlled adversaries fre- 
quently are referred to as "iron Rilot^." 
ThSy' are -computer models that control the 
•actions of simulated adversary aircraft. 
Iron, pilots have been used in yisually 
equipped air combat' simulators such as the 
Air Force SAAC, Northrop Corporation's 
LAS/WAVS, and NASA's Differential Maneuver- 
ing Simulators.^ Prpperly designed, they can ^* ' 
provide realistic 'adyersary maneuvering* 
while unburdening the instructor from con- 
trolling the adversary. When combined ntith ' 
an automated performance measuremer>t££apa- 
bility, summary information can be generated 
describing engagement final putcpme, offen- 
sive/defensive tiptes for each Aircraft, 
time in the gun envelope, time in missile 
envelopes and similar performance infor- * 
matibn. * 



Three primary instructional values are 
associated with computer controlled adver- 
saries. One is the repeatability of the be- 
havior of the adversary, which is consistent 
and predictable (by instructors) during 
training and provides a more consistent base 
line against which' to evaluate student per- 
formance an'd diagndse learning problems, A 
second value is the unburdening of the in- 
structor during training so he -can concen- 
trate on student performance and provide 
more meaningful and timely guidance and feed- 
back, A third value is the lessening of 
specialized instructor skills that must be 
developed to continuously control a simu- 
lated adversary aircraft from a remote con- 
sole. , c * 
' » « 
Iron pilots with, selectable levels-of 
pilot skill hold'considerable potential for 
* " air combat training. Easier adversaries 
could be used earlier in training. As the 
student's Skill levels increase, more dif- 
ficult adversary reactions could be selected. 
The progressive approach to adversary capa- 
bilities could 'hasten the learning process. 
This was the rationale behind the "normal" 
and "difficult" autopilot adversaries d^ 
veloped for the Northrop L/8/WAVS simula- 
• -tion. Instructors in LAS/WAVS used both/ . 
difficulty levels for the training of tran- 
sitioning students (Spring, 1976; Payne, " 
et al,, 1976). 

- . Experience gained .with iron pilots in 
-the SAAC device and the' Northrop LAS/WAVS 
suggests three relevant design considera- 
tion^. The first \% that adversary actions . 
controlled by iron pilot nfodels must be- 
real istic. Original iron pilots were "too ' 
good" and consequently were unrealistic. 
'They operated an perfect information, and 
, their decisions were made^ almost instantly. ' 

This made them virtually unbeatable. ^They 
s had little training values as a result. 
When iron pilots* are designed to provide 
realistic maneuvering,- they are well re- 
ceived by instructors and are used exten- 
sively. The second consideration is that . 
iron pilots with selectable "skill levels" 
should Be developed so that adversary re-., 
sponses can coincide with student pilot 
skill levels during training. Jhe third con- 
sideration is to incorporate fundamental oV 
basic adversary maneuvering capabilities 
(such as simple turning maneuvers) for use 
early fn basic ai^ combat training. .* 

Quantitative Performance Measurement > 

.Quantitative performance measurement 
(QPM)'for training is the computer-based 
monitoring, recording, processing and dis- 
playing of objective,' quantitative infor- 
mation for describing and diagnosing student 
performance. QPM systems have been fairly 



.couindrt ia .research simulators for over ten 
years, but research ^systems are. not well 
Suited for operational training. They have 
been tailored for research use and frequently 
produce voluminous performance data that re-, 
quire subsequent statistical processing. A 
QPM system designed for use in training must 
"perform all statistical and other processing 
of performance data in real or near-reaLtime 
so that students* and instructors are provided 
with useful, concise and timely performance 
feedback information. f 

Practically all quantitative measurement 
capabilities in existing ATDs or ATDs soon t<J 
be delivered are best described as perfor- 
mance monitoring and data recording capabili- 
ties. They alloitf instructors to select tol- 
erance bands (e.g., +/- 100 feet) around 
various performance parameters (e.g., alti- 
tude). The system then monitors for cases 
<that exceed tolerance band values, and re- 
cords ^and/or reports out of tolerance condi- 
tions. Such limited performance monitoring 
capabi 1 i tijes have been applied effectively 
to drive automated cueing 1 and coaching sys- ' 
terns where individual parameter variations 
have beeyi assumed to' have instructional 
meaning. However* "such capabilities have 
found little acceptance for performance eval- 
uation and learning problem diagnosis in 
day-to-day training. In other words, such 
systems are not used by instructors. The 
volume of data produced by. such systems often 
is overwhelming and is difficult to integrate 
# and inten>ret. Also, instructors almost 
'never ar*trairted to use individual parameter 
variation data meaningfully for training. 

APM research has s^own that some mea- 
sures contribute more to the total descrip- 
tion of studep* performance than do others. 
Research also has^shown that, individual mea- 
sures may not be useful for discriminating 
between "good" and "pooir M performance, but 
properly weighted and combined 3re quite use- 
ful for discriminating performance differ- 
ences (Waag, Eddowes, Fuller and Fuller, 
1975). The extent to which various measures 
must be weighted and/or combined remains a 
research issue, but the need to do so has 
been demonstrated for basic instrument flight 
maneuver training (Vreuls, Wooldridge, 
Obemj'ayer, Johnson, and Goldstein, 1976) and 
air combat maneuvering training (Kelly, 
Wooldridge, Hennessy, Vreuls, Barnebey and 
•Cotton, 1979). • - 

. Quantitative measurement of. performance H 
'of procedural sequences is i relatively new 
technology. A number o*f newly acquired -ATDs 
will incorporate procedure monitoring capa- 
bilities (e,g.,,F-16, A- 10, F-5, B-52 and 
C-130). These systems will display the se- 
quences in which procedures are performed. 
It will be the instructor's responsibility 



to determine whether or not the performance 
.acceptablef 

Under Navy sponsorship, bTends of 
"manual" and "quantitative/automated" per- 
formance measurement capabilities wefce in- 
corporated into a recent experimental 
prototype instructional support system 
(Sempie el at., 1979). Among other factors, 
procedural performance was displayed on a 
video display terminal. An ideal sequence 
*'6f procedural steps was displayed, irrele- 
vant procedures were separately displayed 
?s they occurred, and the clock times at 
which alt procedures actually wewe performed 
were displayed beside the steps. In a 
rather limited initial trial use, '"instruc- 
tor pilots found this display valuable, al- 
though *actuaT event times Were not consi- 
dered necessary (Sempie, 1982). 

In the same syst'em, procedural perfor- 
mance scores were derived and displayed. 
The scores were based on algorithms developed 
by a group of highly qualified instructor 
/ pilots; and following much heated debate. 
In practice, the weighted scores were judged 
invalid by instructor pilots. The lesson 
seems to be that vajid quantitative per- 
formance measures ('individually) and scores 
(collections of weighted measures) must be 
derived through statistical analyses, at 
least for flying training. 

Taken together i flying training QPM 
capabilities which emphasize either quanti- 
ties of individual measures or analytically- ( 
derived weightings of several measures will 
be of little practical value until both 
instructor training and measurement method- 
ologies improve with respect>to quantitative 
measure indices. ' . 

Guidelines for the design of practical, 
valid and acceptable QPM systems are not yet 

_ % at hand. Human performance is complex, and 
one human's evaluation of another human's 
performance is more complex. Computer-based 
systems for assessing and diagnosing human 
performance are beginning to evolve, but 

^ operational applications of true QPM systems 
basically are non-existent in flying train- 
ing. 1 Further research is required. 
* 

. INSTRUCTOR TRAINING"" 

Virtually all instructors whcrtrairi 
• other pilots using aircrew training devices, 
dther than basic procedures trainers, are* 
rated^ airmen. Typically they are motivated 
and dedicated personnel who are highly cpm- ' 
petent at performing the tasks they are 
teaching others to do. They may have been , 
assigned their instructional duties, br 
they may have volunteered for the job. 
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•Instruction (in ATDs and/or aircraft) may be 
their primary job assignment, or it may be' an 
assignment dominated by collateral duties. 
Specific training for their instructional 
assignment may have been systematic and 
rigorous, but more likely it was not. 

The best of training equipment, by it- 
self, will* not produce operationally ready 
aircrews. The equipment must be used effec*- 
tively and efficiently to achieve this goal.' 
Obviously, instructor training should be cen- 
tral to the effective and efficient use of 
ATDs.. This comment continues to hatfe con- 
siderable face validity even .though there is 
no empirical evidence indicating that rated 
personnel are required for fUght-orieYited 
. training, or that instructor training in 
instructional principles, ATD use, or in- 
structional feature utilization actually has 
any benefit (Caro, Shelnutt and Speafs, 1982). 
The fact of the matter is, these issues never 
have been systematically examined. 

It seems self evident that ATD instruc- 
tors are central to the proper use of train- 
ing devices. Part of the instructor's job 
is knowirig how to -operate AIBs. A second 
part is knowing how to use such devices §nd 
their capabilities effectively. Achieving 
the second goal requires knowledge of device 
capabilities along with .principles *of in-«, 
struction. Present, typical military in- 
structor training provides neither with 
certainty. 

There are exceptions. • However, typical 
military pilot instructor training programs 
focus on how to perform the tasks to be /t v* 
trained, safety, an$ training- related ad- 
ministrative matters. On average, only about 
three hours of formal instruction deals with* 
how to be a teacher. The operation and use 
of ATDs typiqally is left to informal on-the- ' 
job training. Overall, there is little in- 
structional quality control, except for 
standardization and evaluation functions, 
which may or may not focus on instructional 
, processes and products.* 

There is no question that the military * 
pilot training sy5tem, including simulation 
•training, works. The issue really is one of 
efficiency and effectiveness: could more be 
done arid could it be done with more efficient 
use of resoaroes. Simulation plays an im- 
portant role 1d pilot training, and this role 
likely vyill grow. , As it, grows, instructor 
selection and training will be keys to en- 
hance productivity. Perhap^it is time* that 
the prinpiples of instructional system de- • 
velopment are applied to ther tasks of man- 
aging and conducting training, as well as to 
the tasks to be 'trained- 
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THEX CAN MAKE OR BREAK YOU: CONSIDERING THE INSTRUCTOR 
' AS A USER IN AUTOMATED TRAINING -SYSTEMS 

Dr. Robert Halley 
Logicon, Inc. 



Military systems are becoming more 
advanced and ccnplex. This results in a 
requirement for 'increased training tine by 
users* Concurrently, the growing manpower 
shortage is causing a decrease in the 
number of experienced people available "to 
provide instruction and training on any 
new system. Thus r the military is facing 
a need to provide more training using less 
instructor resources. 

One solution h&s been the development 
of automated training systems to help 
shoulder the training burdens. These 
training systems, more than just simulators 
,or part-task trainers, present design 
problems which operational systems don't 
have* Designers have to produce systems 
which will train the required operational ' 
. skills and also provide training related 
. functions for the sttdents and instruc- 
tors. ..These include such capabilities as 
on-system instruction, performance 1 
measurement, preprograntned scenarios, auto- 
mated feedback, and training management 
functions. 

Training system design is a fairly & 
young art and the Resign artists are slowly 
learning how to do what they intend. In 
the past, the system designers have tended 
to concentrate primarily on hafdware and 
software considerations. More recently, 
there has been an increasing realization 
that courseware considerations (e.g., what 
is being trained, who is being trained, 
etc.) shpuld also be a driving force in 
training system design. The latest reve- 
lation in system design is "peopleware" , 
attention to the people factors in the 
interface between man and machine. 

Unfortunately, for many designers the 
temrf "users" only means students. Training 
systems have many other sets of users. The 
users include instructors, operators, ad- - 
minis trators , and maintenance personnel. 
Each of these sets of people will use the 
system and their needs should be considered 
in the system .design. 'Of this additional 
set of users, instructors have the most 
influence on system implementation and , 
success. As a result, it is very iirportant 
to design the system to be instructor * 
friendly. ' * 

There* are a large number of f actors to 
consider in making automated training 



systems instructor friendly, it is important 
to note that ♦instructors, often for good 
reason, tend to resist a change from their 
traditional approaches to ccnputer-based 
training. First, they see that they will 
have to learn a new job, when they already 
know the old one very well. Second, they 
often perceive this change as a demotion 
from an iirportant instructional position to 
an assignment as a computer system lackey. 
Third, there is a cannon- popular fear about 
having to deal with things conputerish. 
Conputers are typically regarded as being 
mysterious,'* expensive, and very breakable. 
This keeps people from just stepping in and 
using them. Fourth, the instructors have 
real concerns about whether the new system 
will be a training inprovement. They are 
often concerned about both instructional 
effectiveness and possible dehumanization 
of the training environment. 

This instructor resistance can be ex- 
pressed in a variety of ways, anywhere from 
an outright refusal to use the syston to a 
subtle lack of faith in the system which v 
gets transmitted to the student. All these 
types of resistance can lead to the same 
outcome. No matter* how good a system is, 
if the instructors don't like and trust it, 
they can cortpletely destroy its effective- 
ness. A personal exarrple dates back to 
college, when a new videotape-based curric- , 
ulum was beim used to replace normal 
lecturing in an. oral conmunications course. 
The instructor wasn't convinced about. the 
, new approach and. introduced the materials 
fcy saying "All right, you guys' 11 have to 
watch these dumb tapes. Then we'll get to 
the important stuff." Not many people 
watched .the tapes very carefully. 

Implementing instructor friendliness in 
a system means anticipating the instructors ' 
needs and making them as easy as practical 
to accomplish. It is not enough to put 
together a system which can do many and magic 
things and not also make it 6asy to use. For 
exanple, many systems have extensive error 
checking to help identify when problems 
occur, but the system reports themes a "TM 
error 106" or a "100 413 218 error" leaving 
the user to first go look up in an error 
table precisely what happened and then to 
figure out what .caused ,the error. A more 
user* friendly approach would be to have the. 
system report something like "That state- 
meat is incorrectly formatted. 'Please < 
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correct the statement and re-enter it." 

Practically/ you need to get the in-. J 
strvctors -involved as early in the system 
design' process as you can. Instructors can 
provide inportant input in identifying vrtiich 
tasks they will use the system to accomplish 
and what ways those %sks can be done siitply 
and effectively. This information can be 
used in designing the system-instructor 
interface and in preparing the instructor 
training materials. ~In one Logicon system 
design , instructor pilots were utilized in 
designing the system's information- presen- 
tation approach. The result was a very con- 
cise set of CRT displays which have been 
well received as comprehensive and easy to 
use. 

Instructional design is typically ■ 
accomplished by contractors or military 
curriculum designers working outside the 
actual instructional context. They often 
fail to recognize that the instructors have 
inportant things to contribute concerning 
the curriculum and about the, instructional 
approaches and methodologies that are 
feasible and practical for the student 
population. It is valuable to get in- 
structor input and then make sure you point 
out their contribution in the system 
materials. This will add credibility to the 
system and give subsequent instructors seme 
pride of ownership. 

Training systems are Often implemented 
at user catmands with only a minimum of 
acoenpanying documentation. The documen- 
tation is sometimes supplemented by a 
minimum of user training and then the in- 
structors are left on their own to figure 
out the rest. The typical result is that 
the system is exercised in only the 
simplest modes. 

It is important to integrate 'instructor 
training into the system for both initial 
and continuing «use. This ^brportant tool 
can affect thl instructors willingness to 
use the systeqi. It. is important tb -ensure 
the instructors 1 knowledge and attitudes 
about the system. Inportant training topics 
include (1)^6ie^pntinuing importance of 
the instructor's fcples in system s uccess/ 
(2)* the reasons that the system is being 
used to. replace £he old methodologies (3) 
how the system ife used in this instructional 
situation , and how to use the system to maxi- 
mum instructional advantage. Using the 
system to provide this training can also 1 
shpv; the instructors tl^at the system can 
,trai/i effectively. , 



This trailing most provide the rationale 
fbr the development of the system and present 
the system's approach to the instructional ' 
task. . Tte system should also detail the 
inportant decision-making roles that the 
instructor will be asked to fill and also 
explain the breadth of the system's capa- „ 
bilities fo*- instructor options. Practice 
should be provided in utilizing all the 
options "and capabilities. 

» 

The initial instructor training can be 
provided through a thorough instructor 
tutorial which covers all the topics listed 
above in an. instructional package. This . 
should be supported by a system HELP capac- 
ity which provides access to irtfividual 
topics from within the tutorial. This would 
allow a presentation of any area in which 
the instructor needs a review. 

A pragmatic exanple of applying these 
design principles is the Logicon developed 
Instructor Support System (ISS). Designed 
specifically as a tool to assist instructors 
with trainer associated tasks, in its initial 
application, ISS has been attached to an 
existing flight simulator. ISS is being 
used to replace an instructor station that 
is so difficult to use that the instructor 
must spend most of his energy interacting 
with the instructor station, leaving very 
little tine or energy for the task of in- 
- structinq. / 

The ISS provides many functions to 
support the instructors 1 tasks. For exanple, 
the instructors can choose from jthree * 
different ISS training* modes:' specialized 
task training (SIT) , instructor scl^bt v _ . 
(ISEL), and canned. 'Each* of these .training 
nodes allows the instructor to hand tailor 
the students' simulator experience during 
each exercise. The ISS training curriculum 
is conprised of a set of training modules 
such as "afterburner take-off" or "Miramar 
TACAN 1 approach" or "left engine fire." 
Each module is separate and includes sof t- 
* ware identifying what behaviors are to be 
measured, how success is to be graded, what 
checklists and procedures are involved, and 
what marks the beginning and end of the 
task. In each training node these modules 
are utilized differently* 



% STT node allows the instructor to 
schedule one or t more training modules which 
feature practice on a specific -skill. This' 
• allows repeated executions a of a task such 
as landings without having to practice take- 
, offs and other tasks associated with a 

normal mission. Thus, if the instructor . 
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fflt the student needed- work on landings, 
several landings in a row could be scheduled. 
After each" landing the computer would 
automatically set up the next one until the 
session was finished. 

ISEL mode requires the instructor to 
assemble a complete* "chocks -to-chocks" 
mission. In either ISEL or SIT the in- 
structor can choose malfunctions" to be in-""* 
serted into the practice. 

The canned mode provides a predesigned 
-practice -session. In this node the 
training nodules have already been selected 
and sequenced and the instructor need only 
select the overall package. Since canned 
mode practice sessions are** generally de- 
signed to follow the training syllabus, the 
canned mode helps to' ease the instructors ? 
training burden. Canned mode exercises are 
also used in check rides and "graduation" 
exercises to provide a consistent rreasure-* 
ment environment. 

In addition to assisting the instructor 
with his task by providing practice exer- *4 
cise control, ISS provides other instructor* 
support features. These include student*- 
monitoring, performance measurement, 
grading, instructional records' keeping, and 
student* performance debriefing. The student 
monitoring function allows the instructor 
to watch a display of what the student is 
doing while it is being done. Important 



display information including aircraft con- 
figurations (e.g., flaps up, hook down), 
aircraft parameters (e.g., speed, altitude), 
a geographic plot and historic trail of air- 
'craft position, and diagnostic message as 
problems occur are clearly presented on two 
display devices, rather than spread over a 
ten-foot long instructor console. 

The diagnostic messages are a very im- 
portant, feature which appear, thus far, to 
be unique to ISS. Although seme training 
systems have diagnostic messages which are 
displayed at the end of a practice session, 
ISS provides a real-time ^diagnostics display 
at a very detailed level. Ibis allows the 
instructor to know immediately the precise 
cause of a student problem. Figure 1 below 
shows an example of I$S diagnostics. 

The performance measurement function* 
keeps track of student behaviors and com- 
pares them to expected behaviors. The ■ 
grading function uses these performance 
measures to provide a suggested grade -for 
the student practice session on any of a 
number of skills. 'Since there is often., 
some instructor resistance to having the 
grading function taken over by a computer, 
ISS has been carefully designed so as to 
easily let the instructor review all the 
scoring criteria and how the student's .grade m 
has been derived, figure 2 on the following^ 
page shows a sample of the ISS "grade 
sheet." The instructor can review the 
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FIGURE 2. ISS GRADE SHEET DISPLAY 



session arri quickly and siitply change any 
of the grades. The scoring criteria have 
also been designed so that they can be 
easily revised if the standards are incor- 
rect or are changed. 

The ISS instructional records keeping 
function stores "the performance measure*- 
ments, results, and grading for iitmediate 
CRT access and as well as for printing hard 
copy records. She student debriefing 
function is an extension of the records 
keeping capability. The student's last 
entire practice session is recorded and 
the individual tasks can be played back at 
either normal or fast (4 times norrhal) 
speeds or can be frozen at any poifat for 
the student ard instructor to review\ All 
of the aircraft data and student perfor- 
mance dataware replicated 'during the debrief" 
playback. «■ ' - * 

.ISS playback is different than that 
found in most similar systems. In newer 
training systems, for exanple, the play- 
back- node recreates the cockpit activities 
such as itoving the stick and the pedals. 
This information is not very valuable* in- 
struction-ally. .In ISS' the diagnostics 



messages, aircraft parameter displays, and. 
a historic radar and cotntunications trail 
are provided for review. 

The JSS provides multiple functions to 
assist the instructor with his job. Be- 
cause of careful use of instructor input 
and a continuing consideration of the in- 
structor as us**r durina the desiqn process, 
the ISS -provides a very instructor friendly 
interface. The instructions for system use 
are defined in termiroloqy normally assa> 
ciated with the aviation environment. A 
series of decision menus are readily acces- 
sible through -touch, pad controls on th£ lower 
display.' In contrast to man% new systdrs 
(e.g.* 2F112, 2F119) where scenario 'genera-- 
tibn is' very difficult, these menus provide 
the- instructor with complete exercise 
authoring and practice control capabilities 
through a simple series of screen touch 
interactions. 

The ISS encompasses an extensive . ir- 
strtctor tutorial introducing the system 
and the system's functions as well as giving 
the instructor an extensive series of prac- 
tice exercises. The tutorial has been 
subdivided into a set of minitutorials 
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which are accessible at all times through 
a HELP function. 

There are at least two more lessons to 
be learned from the ISS experience* First, 
it is not enough to have the ISS features. 
The features must be implemented well or 
they can even make the system less usable, 
Bor exanple, most new systems provide in- , 
formation on CRTs and provide system con- 
trol through light pens or function keys. 
The concept is good, but the tendency is 
to provide the system access and system 
information without nuch thought to hunan 
factors. Thus, the screens are crowded and 
hard to read and the menus hard to use. 

Second, the ISS is not yet aJLl happi- 
ness and light. It/ too; still has sore , 
problems in the area of user friendliness. 
Design of good practice sessions requires 
that the instructors have done some prior 
planning and know what they are going to 
use the session to accarplish. Otherwise 
the training session can turn out somewhat 
haphazard and not meet its training objec- 
tives. These requirements for planning and 
forethought can make the ISS a little 
scary to the user. Requiring this pre- 
planning may, in the end, be beneficial to 
the process, "but it is not entirely user 
' friendly. An improvement on this approach 
might be to have the system be "smart" 
enough to help the ^instructor develop the 
process by haying guidelines and rules* 
built in. 

Any good training system design will 
attempt to design the training system to 
be sufficiently capable and flexible to 
ensure that the machines serve the users 
rather than the other way around. The % 
users should not have to acccnrnDdate the 
hardware and software by learning little 
tricks and changing their behaviors to 
meet the machine's needs. However, the 
choice qf any approach will constrain the 
ultimate flexibility of the design and its 
ability to meet all of the users* needs. 
If, for exanple, you put the "ON" key on 
the loft, people who are used to having bhe 
"ON" key Qn 1 the right or who have never 
had to turn anything "on" will have to 
learn to adapt to the machine. Money, 
personnel, and tiro constraints tend to 
conspire to make your design choices for 
* 1, but it is inportant to remember to 
ffild in as much user consideration as you 
5an and to learn from your mistakes.. ISS, 4 
in its next incarnations, is going to pro- 
vide a mode where it acfes 5^st like the 
instructor console it is Replacing. ' This 
way the instructors will have to learn 
fewer new skills, but will still have the 
power of the ISS at their disposal when 
they want it. 



It is grucial to remember that instruc- 
tors are a vital instructional resource. 
- Instructors can help a training sypstem pro- 
vide effective and efficient training or 
they can. severely limit the system's useful- 
ness siitply by how they respond to the 
system* Instructor friendliness, then, be- 
comes a major consideration in training 
system design. Two important initial steps 
in building an instructor friendly system 
are getting to knew the instructors 1 needs 
and getting the instructors involved in the 
development process. A third very important 
step is using the training system to train 
and convince the instructors about the 
system's usefulness. A fourth important de- 
sign consideration is building the system 
to serve the user, rather than vice versa. 
Lastly, one must remember the, "user friendly" 
includes all the users, not just the stu- 
dents. 
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INTRODUCTION 

Use of the computer in its vari- 
ous manifestations as a medium for 
instruction has contributed much to 
our understanding of the instruction- 
al, process. Impact of this powerful 
medium on instructor roles, however, 
has received little theoretical at- 
tention. It is^fche purpose of this 
paper tp examine the impact of com- 
puter-based instructipn on tradi- 
tional instructor roles and functions 
in order to achieve a better under- 
standing of the possibilities it 
creates for enhanced instrUctor- 
mach ine interfacing. 

Use of instructional media of 
any kind tends to alter in some ways * 
instructors' roles, at least how they 
are performed. Some media may alter 
the functions performed by instruc- 
tors to such an extent that qualita-' 
tive changes in roles^results. it is 
our contention that such changes 
occur wherever the computer is em- 
. ployed as the primary medium for 
instruction. Wherever it is used in 
whatever mix with other media, it** 
seems that the computer uniquely 
alters instructor functions. 

The typeof computer application 
we shall consider consists o£ a net- 
work of terminals tied to a, central 
processing unit (CPU), in its sim- 
plest form the system may consist of 
a single terminal devoted entirely to 
student use. *;*In more elaborate form 
the system' may contain several termi- 
nals, ^ at least one of which would 
serve as the instructors console. 

In ev£n the simplest form of the 
system, there would exist two-way 
communication between terminal and 
CPU..' In its more elaborate form, 
there also may exist the capability 
for communication among terminals 
directly and via the CPU. Theoreti- 
cally, the inputs to and outputs from 
such a network may include virtually 
any response a human is capable of 



making so long as it can be transdu^ 
ced at the terminal, and any pattern 
of energy change a .human is capable 
of perceiving if it can be displayed 
with fidelity to the human* s' senses 
at the terminal. As futuristic as 
this conception may appeary-su'ch 
networks already form the core of 
multi-terminal' training systems. 
Indeed, simplified versions may be ^ 
found in your child's toy box. This 
is not to say that limitations on 
transduction, processing, and dis^- 
plsys are not real, but rather these 
limitations are currently technologi- 
cal as opposed to conceptual. The 
basic idea seems clear, but its rami- 
fications for instruction do not. It 
is to achieve a better understanding 
of this that we shall attempt to 
explore the conception more closely, 
especially as it applies to the in- 
structor. 

How much of the human instruc- 
tor's traditional -roles may be tak y en 
over by computer-based instructional - 
systems? what functions will remaii\ 
or newly emerge for human instructors 
to perform? These are questions 
especially pertinent to the design, of 
instructor stations. 

For purposes of this paper "in- 
structor "station" is defined as one 
member of a network. qf computer ter- 
minals driven by a CPU 'programmed for 
interactive instructional delivery 
and two-way communication between the 
instructor's terminal and each stu- 
dent's terminal., Clearly, the *role 
of an instructor operating sgch a 
station would differ significantly 
from his.role in the traditional 
classroom even if the instructional 
subject matter remained the samel 

Before changes in instructor ^ 
roles and functions due tovuse of the 
computer as an instructional medium 
can be assessed, it seems*, necessary 
to first arrive at sdme categoriza- 
tion -of them as they havfe been tradi- 
tionally practiced, and to further 
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examine -the major classes of factors 
which determine their selection* 

TRADITIONAL ROLES „ ( * 

Five instructor roles' may be 
distinguished as -a minimum number of 
categories needed to describe the. 
major domains of instructor activity 
within.. traditional settings. These 
are: 

1. Lecture* 

2. Leader 

"3. Supervisor 

4. Tutor 

5. Aide . 

Instructor roles are here dis- 
tinguished in terms of the emphasis 
each gives to the performance -of 
seven categories of instructor func- 
tions. 

TRADITIONAL FUNCTIONS 

The major categories of instruc- 
tor, functions in traditional instruc- 
tional contexts may be divided into ( 
seven categories. These are: 

1. information development 

2. Information delivery 

3. Student supervision 

4. Student guidance 

5. Student evaluation 

6. Data management 

7. Course management 

It will *be apparent in the fol- 
lowing definitions of traditional 
instructor functions that all cate- 
gories are not mutually exclusive of* 
one another. Even though various of 
these functions are interdependent 
and overlap somewhat, they neverthe- 
less represent different channels 
. into which instructional energy is 
directed. Function definitions are: 

" 1. mfdrmation development : The 
gathering, synthesizing, and organi- 
zing of subject matter (facts,, con- 
cepts*, principles, procedures, 'etc.) 
pertinent to 'course objectives into 
formats appropriate tor 'presentation 
in particular instructional situa- • 
tioris. In many industrial and armed 
services situations, more 'formal 
approaches to instructional systems 
development (ISD) would be taken, 
beginning With task. analysis. 

2. information del ivery : ■ The 
presentation of subject matter neces- 
sary to achieve course objectives by 



whatever means that are available, 
effective, and consistent with the 
instructor's role and situation. 

3. Student supervision : The 
direction of student learning activi- 
ties toward timely realization of 
course objectives through mandate, 
directive advice, personal example, 
assignment of study materials, stipu- 
lation of practice techniques,, per- 
formance evaluation, praise, criti- 
cism', etc. 

4. Student guidance : The assis- 
tance of students in discovering, 
orienting toward, and developing 

' feasible approaches to realization of 
future goals through counseling, 
iaterpretative evaluation of apti- 

* tudes, providing information perti- 
nent to formulation and realization 
of long-term objectives, etc. 

5. student evaluation : Assess- 
ments of relative performance, sub-, 
ject matter and/or skills mastery, 
and any other dimension of behavior 
correlated with success in an area of 
activity (attitudes, motivation, o 
emotional stability, etc.) on the ^ 
basis of test scores, proficiency 
scores, subjective ratings, physio- 
logical indices, etc. 

6. pata, management : The record- 

* ing, categorizing and filing of indi- 
vidual performance data; calculations 
of norms, trends, statistics, etc.; 
documentation of instruction-related 
student activities (attendance, 
promptness, awards, demerits, etc. ) ; 
collation of summary data for, and 
preparation of, student progress 
reports; etc. 

* 7. Course management :" Control- 
ling the conduct of a course of in- 
struction inducing all decisions 
regarding subjecf matter, examples, 
demonstrations, .etc. , to be presen- 
. ted, the study materials and practice 
exercisesto be required, the in- 
structional media to be Used, the 
tests and other evaluative instru- 
ments to be used, the sequencing and 
time allotments 'for instructional 
, 1 segments/ the updating and revisions 
J of course objectives, syllabii, *and 
instructional .materials-, etc., the" . 
allocation of instructional resources 
(funds, supplies, etc.) and facili- 
ties, etc. 
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ROLES AND FUNCTIONS 

Rdles are not sharply delineated 
in terms of the traditional instruc- 
tor functions, but rather ,in terms of 
the ways the functions are performed 
and the level of responsibility each 
role assumes for the execution of a 
given function. Not all <*£ 'the tra- 
ditional functions are present in 
every instructor role, e.g., aides 
are usually not responsible for in- 
formation development and lecturers 
are minimally, if at all, responsible 
for student guidance. Even where two 
roles are responsible fpr the same 
type of function, they would rarely 
carry them out in the same manner, 
e.g., the delivery of instructional 
information by lecturers, and super- 
visors is entirely different. 

Both the level of responsibility 
for, and the manner of performing 
each of t,he traditional instructor 
functions may be viewed as outcomes 
of a deterministic process. 

DETERMINANTS OF FUNCTIONS 

There appear to be four immedi- 
• ate determinants of- instructor func- 
tions. These are: 

1 . Instructional objectives 

2. Instructional media 
^3. Delivery situations 

4. Instructor/student ratios 

Usually, the above conditions 
, are established by a more remote 
group of determinants, including: 

1. Instructional goals 

2. Entry-level requirements . 

3. Through-put requirements 

4. .Exit-proficiency requirements 
5/ Resources and facilities 

6. Time/Cost constraints 

While the above by no means 
exhausts the realm of potential de- 
terminants, immediate or remote, they 
seem to be the ones that are most 
influential in mos.t instructional 
settings. Consider the following 
examples: 

Example J.< Assume that the 
instructional goal oi a university's , 
department of philosophy is to offer 
instruction in philosophy of science 
at a. level which could be taken by 
graduate students irf all areas of 
science. Entry-level requirements in 
this case would be quite general, 
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limited to students with graduate * 

status, but stipulating no specific 
, prerequisites. ' pniversity and de- <» 
partmental policies would determine 
the through-put requirements; a mini- 
mum and maximum acceptable/ number of 
students per academic term. Exit- 
proficiency requirements could be set 
; by a committee of experts* consistent 
'with the instructional goal, but 
' usually this wo,uld be done >by the 
instructor (s) . . Resotoces and facili- 
ties would include instructional 
; manpower,' library holdings, media, 
rooms, etc. 

In order to satisfy the instruc- 
tional goal, a high level of exper- 
tise would be required-of the in- 
structor, usually, a Ph.D. with a' 
specialty in the philosophy of sci- 
ence. Few universities would have 
more than one such individual avail- 
able and it is unlikely that the 
university could afford to bring in - 
additional instructors. The bottom 
line would be that this course of 
instruction would be offered by one 
instructor and that the delivery 
situation would be a classroom with 
media support restricted to that on 
hand (slide projector, blackboard, 
textbooks, mimeo'graphed handouts, 
etc.),. This would not be a serious ° 
handicap since, in this case, the 
instructional objectives would be 
conceptual rather than skilled. At 
this point all major aspects of the 
instructor 1 ^ functions have been - 
determined. 

The cost, constraint has limited 
the number of instructors to one and 
thf through-put requirements, to- 
gether, with the available delivery 
situation, has set the acceptable 
number of' students at, say, between 
15 and 30. The resulting Small in- 
structor/student ratio combined with 
a single instructor delivering ab- 
stract conceptual material in a class- 
room- situation with only rudimentary 
media support would mean the ^follow- 
ing: 

a) The instructor's responsi- 
bility for information development 
would be nearly total and probably it 
would have to be carried out person-, 
ally, th£_ result depending heavily on 
thd level of specialized expertise of 
whiph the instructor is capable; 

b) The instructor would bear ♦ 
total responsibility for delivery of 
subject matter information in the 
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^ ass room*, and, due to the limited 
•Support media at his disposal and the 
number of students he must reach, the 
•only mode* available for information 
delivery would be that of the lec- 
ture; 

* 

* c) The very conditions that dic- 
tate the lecture §orm of information 
delivery would pre-empt one-tp-pne; 
student interaction (unless it occur- 
red outside of class) and, therefore, 
would render* negligible any presump- 
tion of responsibility by the in- 
structor for the functions' of student 
. supervision or student guidance; 

** 

df unless the instructor were 
assigned an *ass istant to evaluate 
assignments, grade tests, and keep 
track of attendance, etc., the in-r ; 
structor's* responsibility for the 
functions of student evaluation and x. 
data management would be total. In 
the case that- an assistant were avail- 
able for these duties, the instructor's 
responsibilities could be reduced to 
a minimum depending upon the competence 
of the assistant; 

ej The instructor would assume 
complete responsibility for develop- 
ing the course, syllabus,, determining 

' when and how much time will be devo- 
ted to each instruct ionai- section of 
the course, the nature and. number of 
assignments and tests to be given, 

••selection of textbooks and other 
'reading materials, and all other 
decisions affecting the conduct of 
the course. 

It is apparent that the job 
description which emerges from the 
above enumeration of functions could 
only.be that of a\ lecturer. The 
roles of leader aftd tutor would en- 
taiL similar levels of responsibility 
for information development and de- * 
livery, as well as] for student evalu- 
4 ation and % the management functions, 
* tout the-'levels of Responsibility 
assumed by leaders and tutors for 
student supervision ,and guidance 
would be considerably greater than 
that afforded by the determinants. 

In the example under considera- 
tion, the instructor coiildbe a (dis- 
cussion) leader if the instructor/ 
student ratio were reduced- thus enab- 
ling a more informal mode of informa- 
tional delivery. in order for the 
instructor to serve as a tutor (per- 
mitting v a highly interactive, one- 
on-one mode of information delivery) 



the university's through-put require- 
ments, -resources and restraint allo- 
cations, perhaps eyen its instruc- 
tional goals^ woufd have been quite 
'different. ^Certainly, > requirements 
for student supervision and guidance 
in association with this course of 
instruction would have been far more 
compelling than they were. The point 
is* that, given the assumptions in 
this example,, the only role that is 
feasible within this traditional 
instructional context is that of 
lecturer . 

v 

It should be. noted here that a 
distinction can be made between two 
kinds of ' instructional, leaders. , The 
first kind is that mentioned in the, 
example above, i.e., the discussion 
leader. The second kind is what we 
shall call the field (or team) 'lead- 
er. Both provide instruction for 
relatively small groups of students 
and the levels of responsibility 
assumed for each of the seven in- 
structor functions is approximately 
the same. ( Incidehtally , the ranges 
of levels of responsibility assumed 
by leaders for the various functions 
appears to be more variable than in, 
the case of any other instructor 
role.) Thd chief differences between 
field and discussion leaders are that 
the latter serves within classroom- 
like situations £nd the former relies 
heayily eta personal example, usually 
with some authority beyond that of 
instruction. 

Example 2'. Assume that the 
instructional goal of a major airline 
is to train technicians in the main- 
tenance of a particular kind of jet ( 
engine. The entry-level requirements 
for students to b$ admitted to this 
program include previous training in 
basic electricity theory, circuits 
design and construction, bluepr int 
reading, transformers and motors, 
electro mechanipal devices, micro- 
processor controls and operations, 
etc. The through-put requirements , , 
call for six students to be trained 
in an ll-we£k apprenticeship-type 
training program. Exit-proficiency . 
requirements stipulate tfrat each 
student must be capable of performing 
al3 normal maintenance functions, 
trouble shooting electrical circuits* 
carrying out performance tests ^nd 
measurements, making minor repairs, 
etc.i physical resources and facili- 
ties' are available at a centralized 
company maintenance school. The 
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information delivery situation will 
be a skills laboratory with. such 
media, as engine mock-ups, engine 
/Circuit mock-ups with programmable 
malfunctions, testing instrumenta- 
tion, individual student studycar- 
rells equipped with audio-visual - 
devices, studeht notebooks containing 
job-aides, etc. 

Although the cost for skills * 
laboratory and media 'are consider- 

- able, they are made feasible by con- 
centrating the training" into one « 
well-equipped centralized facility. ; 
Tins also reduces the number of in- 
structors needed to one. The in- 
structor's qualifications include 
several 'years experience in aircraft 
maintenance plus* successful compile- 

* tion qfr'an extensive instructor 
training program conducted by the 
manufacturer of the engine in ques- 
tion^ it may be assumed that the 
company's maintenance training pro- 
gram is a somewhat streamlined ver- 
sion of the manufacturer's instructor 
training program. The instructional 
objectives would consist almost en- 
tirely of concept applications and 

. hands T pn skills. 

*> 

The instructor's responsibilities 
for information development" in this 
case would be negligible, but he 
would be almqst totaMy responsible 
for information delivery, student * 
supervision, student guidance, stu- 
dent evaluation, and data management. 
The instructor's level of responsi- 
bility for course management would be 
moderate since all decisions regard- 
ing sequencing of instructional units, 
testing, evaluation, etc. would have 
been made at the time of training 
>£>i*ogram development. Given this 
array of function responsibilities 
and their determining conditions, the 
role of this instructor could only be 
that of * a supervisor. 

Although this instructor may, 
"from time to time, carry on group 
discussions with his students, his 
role could not. be that of a discus- 
sion leader because the instructional 
objectives 'dictate that he personally 
supervise hands-On exercises In a 
skills laboratory. Neither- could 
this instructor be regarded as a 
tutor because he has no' responsi- 
bility for information development, 
is minimally concerned with general, 
theoretical knowledge, and he is 
required to demonstrate and oversee 
the acquisition of skills by more 



than one studen£v Superficially, the 
supervisor's rote\*ppear s most' like 
that o<f the aide in that the latter 
•might Well work with thksame number 
of students under much t;h€ 
ditions. HoVever, in this example, 
« ar\ ai$e would have little response 
bility for information delivery and 
modCfate to low responsibility for 
student supervision; guidance, and 
evaluation* in this situation an 
Side's responsibility for data manage- 
ment could range between high, and 
low, and relatively little responsi- 
bility would exist for course manage- 
ment. Thus the configuration Of 
responsibilities assumed for in- ' 
structor functions in this example *, 
clearly limits the "choice of roles ±o- 
that of a supervisor.* 

Tn^relationships between deter- 
jnirian£sf$functions, and instructor 
rMes are both interesting and ,com- 
^ plex, involving considerations 'other 
than just those mentioned in the 
above examples'. One such considera- 
tion is the level and'type of exper- 
tise that is generally required for 
each role. Since expertise is usu- 
ally inversely cor related 1 with' supply 
in the market place, within limits 
expertise can be translated into 
d6llar cost. However, when instruc- 
tor' costs are added to the total 
costs of instructional delivery and 
the cost per student is calculated, 
the 'cost bf instructor expertise in' 
some roles may be markedly dimini- 
shed;' While a highly qualified lec- * 
turef may be expehsiv& relative to a 
supervisor, the cost p,er student for 
the lecfeure-type instruction usually * 
would be less due to the larger num- 
ber of students and the absence of 
any need for expensive media. Even 
so, the matter of instructor exper- 
tise seems to be more a problem of 
.supply than, cost. In fact it appears 
that, the higher the level of re- 
quired expertise is, the' shorter the 
supply of qualified instructors. 

Instructor availability is one 
factor that is directly impacted by , 
modern instructional technology, 4 
especially computer-based instruct 
tidn. If the supply of instructors 
in any given field is- reduced or even 
limited by the, time it takes to 
train, or educate", those instructors, 
then '~any' approach to instruction 
which can replace at least some v in- " 
structor functions with automated •] 
functions should decrease the time 
required to produce" the needed 
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supply.. ..This is npt to say thai: the 
'number of months or years it takes to 
train effective instructors is the 
only factor* affecting their supply at 
arty point in time. Monetary and 
qaree'r incentives, job satisf act ibn, 
aptitude and previous education are 
among the major factors which control 
the number of individuals who are 
motivated and competent. to receive 
instructor training in the first 
place. In this area also, compu- 
ter-based systems should improve 
supply through incentives and job 
satisfaction. Time savings in train- 
ing^and increased motivation, of 
instructors are two factors which « 
shjbuld prove especially important to 
instructional systems' that experience 
$igh rates of instructor, turnover, 
such as those in the^xmed- services. 

It is {tot the purpose -of .the 
present paper to examine the ways in 
whicft computer-based instructional 
systems provide motivational incen- 
tives and increased job satisfaction, 
for the instructors that operate 
them,* but some of these ways will 
become apparent* later on in this 
paper. ""Suffice it* to say that the 
possibilities for instructor motiva- 
tion inherent in the network-type 
computer-bas'ed system is an exciting 
and . important frontier ;for future - 
development. It is the" other avenue 
of impact that the computer-based' 
system- has on instructor supply (in-c 
deed*, instructor effectiveness also), 
with which t we are 'concerned here, 
ire./ the alteration of instructor^ 
functions. * . ■> * 

Which of the traditional* func- 
tions will be^Flt&r.ed? ' Each of the 
functions will be^ altered in some 
ways, and several may b£ virtually 
eliminated. However, one further 
aspecfe^f traditional instruction 
need^Fsome mention before proceeding 
to an examination of automated in- 
structor functions. That is < the 
relationship between* traditional 
instructional delivery situations and' 
instructor roles. 

TRADITIONAL SITUATIONS 

1 The physical situations in which 
instruction is traditionally de- 
livered constrains instructor func- 
tions an& thus narrows the choice^ of 
roles appropriate to be performed in 
them. Five 'major classes of instruc- 
tional delivery situations may be' 
distinguished/ These are: ' : 



1 . Classrdom 
• 2. .Skills laboratory 

3>"^IndividuaLizea study-station 

4/T~' Simulated^ operational en- 
vironment* - 

5. Actual operational ( environ- 
ment 

It is-probably unnecessary , to 
enumerate the distinguishing chairac-, 

♦ teristics of each of these situations 
since they are generally familiar. 
Certainly, this classification scheme 
could be expanded if. ; -finer distinc- 
tions were made. The Scheme offered . 
here is the minimal number of cate- . 

* gories of situations in whicfi in- 
struction is traditionally delivered m 
a*nd which permits the distinctions we 

'wish to make. 

The degree and kind of interac- 
tions between students and instruc- 
tors differ in these situations, 
i.e., the emphasis on functions and 
the way they are carried out change 
as do instructor roles. Instructors 
simply do not lecture in situations 
3, -4, and 5. Only rarely would a 
lecture occur ♦ in situation 2. It * 
seems that the lecture form of in- , 
structional delivery i's largely con- 
fined to situation 1. Thus it goes 
without saying that the lecturer 'is* 
-•at*-;home only* ih the classroom- Like- 
wise, the discussion leader ^also 
finds his primary place there. The 
role 'of field leader, on the other 
hand, seems to occur with greatest 
frequency in actual operational en- 
vironments, but this role would not 
be uncommon in simulated environ- 
ments. Of course, the supervisor is 
most at home in the skills laboratory 
but he may be found in situations 4 
and 5. • Tutors, which appear to be *_ 
decreasing in their frequency of 
appearance, probably due to their 
high cost per student, carry out 
their functions only in individual- 
ized-^tationS'* By contrast, aides 
appear to be the most ubiquitous of 
instructors, occurring. in all situ^ 
ations except the. classroom. 

The changes in. roles with situ- 
ations reflect changes in other trar 
ditional determinants as well. In- 
structional media, objectives, de-- 
'livery techniques, and instructor/ • 
student ratios also change with .situ- 
ations. The 'covariance of these 
immediate determinants of instructor 
» functions is attributable to the 

combined influences t of the condition's 
referred toJ.n the last section as 
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% remote determinants. The flow of 
> influence between remote and immediate 
determinants, however, is not always 
unidirectional, i:e*, from remote to 
immediate to functions and ultimately 
to roles. Rather, the flow of influ- 
ence occasionally is two-way, in both 
the vertical and horizontal directions. 
For example, if the situation we have 
is a one-room schoolhouse, just money 
enough for one instructor who must 
meet state qualification standards, a 
blackboard, a couple boxes of^chalk, 
a closet full of, dog-reared books, and 
^ 30 or so right-handed chairs, then 
the instructional goals and other;' 
remote determinants have to be modi- 
fied bo suit the more immediate on§s, 
if, they are considered at all. It is 
inescapable that, in this 'situation, 
the instructor is going to be a lec- 
turer regardless of what behaviors 
are' to be trained (taught). 

If there is anjj truth to the 
assertion that the behaviors which 
can be trained in any^ situation de- 
pend oh the behaviors* whiSh can bje 
brought to occur there, then it must 
also be. true that instructional de- 
livery situations substantially in- 
fluence training effectiveness since 
they certainly limit (if not induce) 
much of the behavior that does occur 
ift them. Unfortunately, too much of 
what appears to occur in the school- 
room today, seems r to be learned, the 
surprise being that anything else 
Ce.g., academios) could be.. An an- 
swer^ to this problem that one so 
often .hears calls for "more disci- 
pline." It would seem that this 
sentiment is, if nothing" -else, at 
least in the right direction for it. 
implies control of behavior^. We 
suggest that it is the interactions 
With subject matter that needs most-* 
to be controlled. It is just this 
kind of control that automated sys- 
tems can be effective in providing* 

AUTOMATED FUNCTIONS 

In this section we summarize 
what, in general, a* computer-based 
instructional system can provide in 
the place of each of^the traditional 
instructional functions. The lists 
of automated functions presented here 
are state-of-the-art. As exciting as \ 
current capabilities are for instruc- , 
tional application/ it should-be kept ' 
in mind that considerable progress * 
remains to be achieved in all areas , 
of .this man-machine interface (re- 
sponse transduction, intelligent 



programs, information displays) . 
Although some of^tfl^ automated func- 
tions 1 istedb baiow can be performed 
by relatively 4mall ; systems, the type 
system we are considering consists of 
a network of sophisticated ^terminals 
(both student and instructor) driven 
by an imaginatively, programmed large- 
capacity cpy. 

.* 

1 • Development of instructional 
information ; (Tj serves as a guide to 
instructional development by means of 
programs (menus with prompts and 
messages) based on algorithms for 
each successive ,stage of course de- 
sign; (b) facilitates writing,' edit- 
ing, and drawing of instructional 
materials through programs for word 
processing and graphics production; 
(c) permits convenient filing, cross- 
referencing, and combining of in- 
structional information through pro- 
grams for .information management. 

2. Delivery of instructional 
information : (a) presents course 
subject matter ranging from abstract 
concepts to factual itemizations in 
self-paced, Tnastery-based formats 
displayed in written, spoken, and/or 
, graphic forms; tb) delineates rele- . 
vance and applicability of subject 
matter by presenting contextural 
information and sample problem solu- 
tions, problem-solving exercises, 
etc.;* (c) provides instructions /or 
proceeding through programmed les- 
sons, performing skills, correcting" 
mistakes, or obtaining remedial in- 
- formation for review; (d) produces or 
controls simulated representations of 
operational ^devices, field or job 
situations, abstract processes, per- 
formance procedures, job aides, etc, 
enabling demonstrations, practice, 
rehearsal, etc., of concepts, rules/ 
skills, procedures, attitudes, roles, 
team exercises, "what if" explora- 
tions, etc.; (e) delivers individual 
and/or „group response-contingent 
feedback designed to aide self-^3iag- 
ntfsis of learfiing progress; informa- 
tion may consist of prompts, ques- 
tions, encouraging messages, scores, 
etc. , x in a variety of display .formats 
that may include special auditory or 
'visual effects; (f) provides summary 
feedback at the end of each instruc- 
tional unit or major exercise con- 
sisting of scores, outcome state- 
ments, evaluations of relative per- 
formance, recommendations for im- 
provement, overall course performance 
profiles, course grades, etc. 
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. 3. Student supervision : (a) 
guides students along instructional • 
paths adjusted for level. of achieve- 
ment ai\d rate of progress; {by opti- 
mizes fndividua-lized instructional 
paths through frequent performance^ 
checks, variable- path branching, and 
"review Sequences; (c) provides super- 
visory instructions and feedback 
(written, spoken, and/or visual; 
graphical or simulated visual demon-", 
strations of "do this") contingent 
upon individual or group actions with 
possible instructor intervention; 

4. Student guidance : (a)'pro- 
. vides recommendations for future 

courses of action (remedial study, 
more in-depth study, information 
sources, job possibil it ies, etc. ) 
based upon performance profiles; (b) # 
-response to student questions about, 
their performance profiles with in- 
terpretive answers and comparative 
data; (c) provides job descriptions 
(requirements, work conditions, sal- 
ary dtfta, etc.) in afeas related to 
course of instruction. 

5. Student evaluation :' (a) 
determines correctness of choices, 
problem solutions, or actions on an 
item-by-item basis; (b) tests per- 
formance proficiency relative to 
instructional objectives, group 
norms, and standards; (c) diagnoses 
learning progress, detects learning 
difficulties early, prescribes reme- 
dial work, and adjusts difficulty 
levels/rates to match students' a- 
bilities; (d) provides overall per- 

• formance profiles^ course grades, 
percentile ranks, etc. 

• 6- Data management : (a) accepts 
as inputs any properly computer- 
interfaced responses by students and 
instructors; (b) automatically- 
records in central memory input data 
from all system terminals, pools 
group data, forms generic data bases 
and carries out statistical or other 
processing while preserv ing ' indi- s 
vidual'student records; (c) displays 
selected data files, analysis results, 
or interpretative messages auto- 
.matically, or on command to designated 
terminal in written, spoken, and/or 

graphical formats. . 
» 

7 . Course management : (a) de- 
scribes syllabus-controlled sequen- 
cing of, and time allocations for, 
successive instructional units and v 
the, timely exeCutiog of tests, evalu- 
ations, and student feedback? (b) . 



tracks individual and group progress 
relative to' established course mile- 
stones, performance standards, and 
through-put rates; ic) carries out 
cojarse evaluations and recommenda- 
tions* for revision based on data 
analysis and success in attaining 
instructional objectives; (d) enables 
monitoring of individual or group 
performance by instructor who may 
selectively intervene or interact • 
with individuals or group; (£) pro- . 
vides prompts and messages to in- 
structors to insure timely occurrence 
of non-computer instructor functions.. 

c The above lists of automated 
functions include no mention of ^how 
they are, or might be, effected. 
Even though this question' is cer- 
tainly beyond the scope of this paper, 
there are at least three good reasons 
why it is worthwhile to consider 
automated functions independently of 
the technological means^by which they 
can, or might be, achieved; (a) appli- 
cations of technology nuist be justi- 
fied in terms of the functions it can 
perform, and thdse functions must 
stand*on their, own merits; (b) the 
same functions may be accomplished in 
different ways depending on the re- 
quirements of particular applica- 
tions, and (e) new applications of 
the* same functions and new techniques 
for effecting them may be developed/ 
\ it is also the case that new tech- 
nologies result in the emergence of 
previously- unanticipated functions 
\ and applications. As we shall show, 
\ applications of computer technology 
\ in instruction modifies instructor 
\ roles not only by assuming and im- 
\ proving upon traditional instructor 
\,func£ions, but also^by laying the 
^foundation for emergence of new in- 
structor functions. But first, let 
'lis compare automated and traditional 
instructor functions. 

FUNCTION DIFFERENCES ' > 

\ Perhaps the appropriate question 
ao this point would be, what instruc- 
tor functions can, and must, humans' 
perform that computerized machines 
carinot? In answer, we must admit to 
igriorance. It is the word "must" in 
the\ question'" that renders its answer 
obsture, at legist to us.- Even^if we 
cou^d enumerate every possible human 
activity that might be. .construed to 
be an instructor function, we could 
'not say on the Basils of ss4 s entific 
.evidence which of them were^Ssent ial 
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to the learning process. Perhaps 
none are. 

It would seem safe to conclude 
from the last half-century^ research 
on the learning process that the 
minimum necessary conditions for 
learning, are. (a) the presentation of 
energy-borne information toHrhe senses 
of a living organism, and (b) the 
consequent occurrence of some sensory 
effect 'within the organism that may 
be overtly manifested in its behav- 
ior. While these two conditions "are 
necessary, even if they are met the 
occurrence of learning is not as- 
sured. Information enhancement, 
repetition of its presentation, re- 
sponse-feedback, etc., 'also may .be 
required in certain circumstances td 
increase the probability that learn- 
ing w w ill occur, that it will occur 
rapidly, and that it will endure. 
Whether or not .such conditions are 
essential to the learning process in 
some fundamental theoretical sense 
(if, indeed, there is just one learn- 
Y ing process), it see-ms they must-be 
Aconsidered practically indispensible 
us^ircumstances appropriate for the 
accomplishment of instruction. Since 
instructor functions traditionally 
have been the means through wh ich 
production and manipulation of these' 
conditions for learning, has been 
effected, it seems reasonable to 
assume that any instructor function 
which ,a computerized machine cannot 
be -programmed to simulate effectively 
should be considered a "must" for the 
human instructor. Some insight into 
what these "must" functions should be 
may be gained 4 by examining the dif- 
ferences between instructor and auto- 
mated functions. Function differ-* 
ences are summarized below using the 
same numbering of categories as be- 
fore. 

1. Computer-based systems can 
facilitate the development of in- 
structional information, but they 
cannot recognize instructional needs, 
originate program goals, or conceive 
of the means by which to attain them. 
Humans must assume virtually all 
responsibility for gathering and 

- synthesizing information, evaluating 

- what would be important and interest- 
ing, creating stimulating cerftceptu- 
alizations, examples, etc., of the 
information, and finally, originating \ 
innovative computer programs to \ 

•achieve effective information delivery 
to, and interaction with, students. 
However, these indispensible 



instructor functions do not have to 
'be carried out by the .individuals who 
serve as instructors with computer- 
based "systems. In, fact, it would 
probably be mor'e cost-effective for 
information development functions to 
be exclusively the jurisdiction of 
professional scientists, scholars, 
computer programmers, instructional 
designers, etc. The salience of this 
specialist approach to information 
development is ramified in the rapid- 
ly* emerging areas of computer-based 
imagery, animation, simulation gam- 
ing, etc. If the possibilities for 
computer-based instruction are to be 
exploited to the fullest in the years 
to come, it is apparent that we shall 
have to rely more upon the specialist 
(better still, teams of specialists) . 

2. It is in the performance, of 
information delivery functions that 
the computer-based instructional 
system proves its worth. The degree 
to which such a system can carry out 
the automated functions listed pre- 
viously depends on the fidelity of 
its ifian-machine interfacing -^sensors 
and manipulanda; visual, auditory, 
etc., display devices), its CPU 
capacity and the sophistication of 
its software.' If we-wish to think 
very far ahead, display functions 
might be expanded to include computer- 
driven machine movements (robotics). ; 

It is apparent that we can ex- 
pect dramatic* : advances in all areas 
of computer technology in the rela- 
tively near future. However, even 
current capabilities permit the exe- 
cution 6f nearly all information 
delivery functions for which human 
instructors are— responsible, includ- 
ing some functions unachievable by 
any means other than the computer 
(e.g., interactive video displays). 
This is not to say that computerized 
machines duplicate human actions in 
carrying out the same functions, or 
that such duplication is always de- 
sirable even when it is possible. 
For example, it may be that some 
information car be transmitted to a 
student more effectively in a writ- 
ten, visual display than in a spoken, 
auditory display, where the latter 
might be the format traditionally 
used by human instructors. Converse- 
ly, if it were beneficial to transmit 
the information in a simulated human 
form, a speech synthesizer could be 
employed. Thus, in order to dupli- 
cate human functions, it may not be 
necessaryVto simulate huiran actions. 
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A review of the capabilities 
listed under automated functions y 
indicates a high order of individu- 
alized and interactive information 
delivery by computer-based instruc- 
tional systems. The question 'is, are 
there any additional delivery func- 
tions that must be performed by a 
human instructor-operator whose sta- 
tion is a terminal tied into a net- 
work of student stations? It is 
implicit in this question that the 
instructor 1 & communication with stu- 
dents is limited to just those chan- 
•nels available in his station. These 
may be of t<wo types: (a) extra-CPU, 
and (b) intra-CPU. Ey means of the 
extra-CPU channels, the instructor 
may deliver ncn-computer-based infor- 
mation (e.g., speech and'other accou- 
stic signals, visual signals and 
displays, etc). These displays may 
be presented either independently of, 
or in synchrony with, CFU-produced. 
information. Also, the instructor 
may selectively open channels to one 
or more student stations, enabling 
two-way video and/oif audio communica- 
tion. By means of the intra-CPU 
channels the instructor may interact 
in ways and at times permitted by the 
program (e.g., as a" team leader , as a 
competitor in a gaming situation, as 
a prompter, etc.! or he may override 
the program and, initiate appropriate 
pre-program directives, messages, 
data listings; etc. 

. 'These are some of the delivery 
* functions an , instructor may perform 
from within a network station. Due - 
to this system's large capability for 
-performing automated functions, prob- 
ably none of the possible human in- 
structor information delivery func- 
tions £an be considered a priori to 
be indispensible. Depending on spe- 
cific training program requirements, 
both extca- and intra-CPU instructor 
functions may be considered desirable 
features and incorporated as part of 
the whole delivery systert. Since 
„non(* of the automated functions need 
be sacrificed in order to accomplish 
this, it seems highly likely that 
instructor functions of the sort 
indicated above would be included 
wherever possible. In addition to 
monitoring students, inclusion of 
these functions would enable the 
instructor to interact with his stu- 
dents on an individual or group basis, 
competing with or leading them, cor- 
recting or encouraging them, di- 
recting or questioning them, e'tc. 
The motivational influences of these ( 



kinds of interactions on both in- 
structors* should magnify the effec- 
tiveness of the entire learning ex- 
perience in this situation* 

The instructors responsibili- 
ties for delivery information outside 
of the network station probably should 
be regarded $s essential. They would 
include familiarization of students 
with training systems operating pro- 
cedures, demonstrating system opera- 
tion, orienting students to training 
objectives and performance expecta- 
tions, carrying On post-training 
session discussions with students, 
etc. These functions would probably 
require that the instructor serve in 
several traditional , roles: as -lecturer 
during initial familiarization; as 
supervisor during demonstration of 
system operations; and as discussion 
leader during pre- and post-training 
sessions. By contrast, during system 
training sessions the instructor's 
role would be characterized by a 
mixture of elements from the tradi- 
tional roles of field (team) leader, 
supervisor, and tutor. ' An additional 
set of elements of *a non-traditional 
nature may a^Lsd be included, i.e., 
those of a competitor. The compound 
of these elements results in the 
emergence of a new role for the in- 
structor station. For want of a 
better name for this role, we shall 
.call it "coach". 

3. student "supervision is f 
category of functions which ulti- 
mately depend on the delivery of. 
information from instructor to stu- 
dent. Although this information is. 
instructional, the . knowledge compo- 
nents of it are generally limited to 
system, or machine, operations con- 
veyed through demonstrations of the 
skills involved in performing the 
operations. Supervisory information 
3lso consists of evaluative feedback 
• from the instructor regarding both- 
the actions made in skills perfor- 
mance and the outcomes or products of 
those actions. Characteristic of 
supervisory information is a strong 
directive component which is intended 
to control both actions and outcomes. 
As the instructor's responsibility 
for supervisory functions depends On 
how closely the students 1 actions are 
to be controlled. As indicated in 
the list of automated supervisory 
'functions, the computer-based system 
enables a high order of student di- 
rection on a response-by- response 
basis. The instructor is thus 



relieved of mgch of the drudgery of 
student supervision, this being re- 
placed by finely attuned immediate 
"response-contingent feedback to the 
student. Supervisory functions re- 
maining for instructors to perform, 
es in the case of information de- 
livery functions, must be divided 
into station and non-station, func- 
tions, the former being broken down 
into extra- and intra-CPU functions. 

While operating from his network 
station, the instructor's supervisory 
responsbilities would require moni- 
toring of student performance via 
both ComputerLand non-computer dis- 
plays. Instructor supervision via 
intra-CPU channels might consist of 
program overrides, re-initializing 
some portion of the program, inter- 
acting with students in ways permit- 
ted, by the^program (e.g., entering 
instructions, calling up visual dis- 
plays, manipulating pointers and such 
on visual displays; etc.), activating 
pre-programmed directives, etc. By 
means of extra-CPU channels, the 
instructor might issue commands, 
instructions, corrections, etc., at 
any point independently of CPU-con- 
trolled actions. Whether or not, and 
the degree to which, any of these 
activities were considered necessary' 
would depend on particular program 
requirements. The instructor's super- 
visory duties outside of his network 
station, and after initial familiari- 
zation and demonstrations, probably 
would be negligible consisting mainly 
of informal advice and encouragement.' 
Thus none of the instructor's super- 
visory functions can be considered a 
Priori indispensable, though certainly 
they would be desirable in most cases, 
especially those involving skill 
performance . 

It is interesting to observe 
that, even if the instructor 1 s > super- 
visory functions are strong, in this 
type of automated instructional situ- 
ation his role would not be that of 
the traditional supervisor. Rather, 
it would be that of a coach as indi- 
cated earlier. On the other hand, if 
supervisory responsibilities were 
negligible, the instructor's role 
c would tend toward that of the tutor 
(if he supplied knowledge information 
interactively) or that of the com- 
petitor (in a gaming simulation). 

4. The advisory and counseling 
responsibilities usually subsumed 
under student guidance probably would 



be largely peripheral to the instruc- 
tor's network station functions. The 
instructor's responsibility for stu- 
dent guidance outside of the network 
station, however, would include both 
formal and informal discussions re- 
. garding student career goals, rele- 
vance of the current training to 
attainment of those goals, job avail- 
ability and . requirements (some of 
this may be available at the stu- 
dent's network terminal), etc. Fur- 
thermore, specific advisory or coun- 
seling sessions probably would be 
scheduled to correspond with major 
course milestones. , At such times 
students may be faced with decisions 
' involving changes in job aspirations. 
The instructor's counseling respon- 
sibilities probably should be-limited 
to providing recommendations for 
alternative courses of action for 9 
students to pursue depending on their 
success in the present cogrse of 
instruction, formal counseling by 
instructors might be carried out in 
conjunction with, or referred en- 
tirely to, a professional counselor 
knowledgeable in the area of instruc- 
tion. Thus, the computer-based sys- 
tem of instruction under considera- 
tion would seem to permit a large 
range of instructor commitment to 
student guidance and, consequently, 
this may not be regarded as an indis- 
pensible area of instructor func- 
tions. At least some' modest level of 
informal counseling, however, would 
appear to be beneficial. 

5. As indicated in the list of 
automated student evaluation of func- 
tions, the Computer-based system 
requires little additional instructor 
input in this area, if any at all. 
However, if the instructor's super- 
visory responsibilities within his 
network station constitute a signifi- 
cant element in the training program 
the instructor's responsibility for 
some forms of evaluative feedback to 
students would be greater. Such 
feedback would be in the form of 
corrections, reprimands, indications 
of performance deficiencies, etc. 
Likewise, because the system can be 
programmed to provide the student 
with virtually any form of evaluative 
performance index, diagnoses of learn- 
m . ing progress, overall performance 
profiles, etc., the instructor's 
responsibility for student evaluation 
- outside of his .network station would 
"consist largely of interpretation and 
explanation. Of course, need for the 
latter would depend on the ease with 
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which students could read and inter- 
pret their machine-generated evalua- 
tions. In this, as in any other area^ 
6f comptfter-rbased instruction, pro- 
gramming inadequacies devolve into 
instructor responsibilities, in sum- 
mary, 'the network system instructors 
student evaluation' functions are 
negligible except where supervisory 
responsibilities are high. In that 
case, the two are'as of instructor 
functions are parallel and, to some 
extent, indistinguishable. 

• 6, In the area of data manage- 
ment, instructor functions are elimi- 
nated, instructors may enter their 
comments, evaluative scores, jetc.-, 
i/ito student records, but the in- 
structors bear no responsibility^for + 
filing, updating, or other processing 
of performance data. 

7\ 'The mechanics for course' 
management are all pre-set in the 
training program, instructor respon- 
sibilities in this area consist main- 
ly of keeping the prearranged program 
on. schedule, ensuring smooth inte- 
gration of non-program activities' 
with the schedule. Thus, insofar as 
course management is concerned, the 
instructor's role is essentially that 
of an aide. He must service and 
otherwise work to ensure the uninter- 
rupted continuance of the program'. 
In a sense, the program manages it- 
self. This is to say that all man- 
agement decisions regarding schedul- 
ing, resource allocations, program 
requirements, contingency plans (in- 
cluding provisions for instructor 
intervention), etc., are made prior 
to implementation . 

INSTRUCTOR STATIONS 

The kernel of our analysis is 
that, if computer-based automated 
instructor functions are utilizec^&N 
their fullest potential, no humai V_^ 
instructor functions may be consia^r— 
ed a pr ior i as indispensible. The 
machine is mudh mqre than an extra- 
ordinary medium for the delivery of 
instructional information. It is 
al-so a capable instructor. 

The question becomes, then, what 
human instructor functions may be 
considered desirable? A cursory 
enumeration of these functions was 
given in the preceding section. They 
were divided into two situational 
classes; functions to be performed in 
the instructor's network station, and 



functions to be performed outside of 
the station. Within-station fun<?* 
tions were further divided into extra- 
CPU channels and intra-CPU channels. 
The question now that we have to 
address is what characteristics of 
network instructor stations are nec- 
essary to permit the performance of 
human instructor functions? ■ 

The model of the computer-based-- 
instructional system (trainer) that 
we are considering consists of a 
network' of terminals tied to each 
other through a CPU and through di- 
rect circuits... The CPU assumes the 
lion's share of responsibility for 
presenting' information, collecting' 
performance data, data management, 
• interactive processing of inputs and 
outputs, sequencing, etc. , depending 
on the ways 'it is programmed, the CPU 
may call for instructor actions, 
decisions, feedback to students, 
etc., thus structuring and integra- 
ting instructor functions into the 
complex flow of events. If'-the pro- 
gram involves a gaming simulation, 
the instructor may interact, with 
students on a competitive basis as an 
opponent or as a team leader. Fur- 
thermore, through the CPU the in- 
structor's station may be tied into .a 
network of instructor stations there- 
by placing at the instructor '-s dis-; 
posal a generic data base built ujd 
through common experience with tlhe 
system or originally designed for 
instructor training. ' 

It should be clear that the core 
of this system is the CPU, not the 
instructor. Its programming, capa- 
city, and speed determine what it can 
process {the inputs it can accept, 
the decisions it can make, and the 
displays it can present). The in- 
structor station, as well as student 
stations, are the peripheral input 
and output .interfaces with the CPU. 
Excluding direct Links between sta- 
tions, the specifications for station 
input sensors and output displays • 
must conform to CPU capabilities. At 
present CPU technology appears to be 
more advanced than sensing and dis- 
play technology. With some temerity 
we suggest, therefore, 1 that .anything 
which can be sensed in analog and 
converted to digital, and vi'ce versa, 
is within the capabilities of exist- 
ing CPU technology and thus can be 
incorporated into student and in- 
structor stations, ignoring tech- 
nological limitations due to such; 
problems as digital conversion of 
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analog inputs and ouputs, etc,, let 
us- examine some desirable input and 
display characteristics of instructor 
stations. 

Interface features of the in- 
structor station may be organized 
into a two-by-two contingency table. 
The columns of this table designate 
channel type and the rows designate 
interface type. The twd channels 
are, as before, intra-CPU and extra- 
CPU. The two rows are controls and 
displays. Thus we have controjs an£ 
displays for each channel, resulting 
in four categories of instructor 
station features. 

k 1. Intra-CPU displays ; there 
are two glasses of displays within 
this category: (a) monitors and (b) 
read-outs. The intra-CPU monitor 
displays provide the instructor with 
duplicates of the CPU outputs to 
student stations. These would in- 
clude both visual displays (CRT) and 
auditory displays (headphones) . The 
student station being monitored would 
be selectable from the instructor 
station, as would the type display. 
Since these displays duplicate those 
in student stations, they could be 
used by the -instructor while partici- 
pating in game simulations: ^ The^ 
read-out displays would provide the 
instructor with private CPU outputs 
thereby enabling him to receive in- 
formation from the computer indepen- 
dently of that available at student 
stations. Status reports on all* sta- 
tions, data files, prompts and mes- 
sages, etc., would be delivered at 
these displays. 

2. Extra-CPU displays * these 
displays permit two-way communication 
between students and instructor. 
They may be audio (headphone) and/or 
video ( television) displays. In 
addition to communicatiort between 
students and instructor, these dis- 
plays could also be used 'as extra-CPU 
monitors. In either case, Student 
stations would be selectable by the 
instructor, as would open channel 
communication with all stations (audio 
only) . By means of one of these 
display* (probably audio) , students 
would be able to initiate communica- 
tion with the instructor. 

3. Intra-CPU controls : Thesfe 
instructor response interfaces are of 
two types: (a) operator and (b) mas- 
ter. The operator controls duplicate 
those present in each student station. 



They permit the instructor to engage 
the -system as a participant (competi- 
tor or leader) ih game' simulations, 
etc. The master controls enable the 
instructor to address the CPU inde- 
pendently of student stations. By 
means of the master control terminal, 
non-automated instructor functions 
provided for in the computer program 
could be carried out. 1 

4. Extra-CPU controls : these 
controls include "all switches, knobs, 
levers, etc., necessary to activate, 
select, operate, etc., any non-CPU r 
devices or ^displays. Instructor 
controls for extra-CPU displays would 
be included in this category. 

Of course, specific design fea- 
tures of instructor- stations would be 
determined by the same sor'ts of de-~ 
terminants that were shown to influ- 
ence traditional instruction^ In- 
deed, the entire network system would 
be impacted by these factors. There 
is, however, an additional element 
which must Se taken into account in 
designing the instructor station. It 
is the CPU; the sophistication of its 
programming, the capacity of its 
memory, and the speed of its opera- 
tion. • 

* * < 
The technology 'exists to provide 
advanced CPU hardware as it does for 
controls. and to a lesser extent for 
^displays, but the program software 
has to be created for each applica- 
tion of this .technology. We suggest 
that system design and software de- 
velopment should be parallel efforts, 
both directed toward the same end, §n 
integrated instructional system. If 
the contributions of instructional, 
software, and hardware designers are 
weighted equally throughout the plan- 
ning and development stages of com- 
puter-based instructional systems, 
then the instructor station concept 
may be fully realized. At that point 
* we will see the new instructor roles 
emerge; the coach, the competitor, 
the team leader, etc. Whatever kind 
of animal we build, let us design it 
so that its appendages and its brain 
form an integrated whole. In that 
the instructor will be but one func- 
tioning organ . 
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INTRODUCTION 

- The emerging Navy Undergraduate Pilot. (VTX) 
training system, designed under the guide- 
lines of OMB Circular A-109, is a training 
system composed of a number of integrated 
elements. These elements are the aircraft, 
simulators, training ma-nagement system^ 
academic system, and ao integrated logistics 
system. In the McDonnell-Douglas/British 
Aerospace VTX system, the simulators are in 
reality a family of training devices which 
provide a systematic, hierarchlal method of 
training hands-on learning objectives. The 
purpose of this paper is to present the 
front-end analysis procedures which were 
used' during design of the instructor 
stations .for the family of VTX training 
devices. 

PRELIMINARY ANALYSIS r 

The VTX -training system (VTXTS) development 
effort centered, ^cound^ai^cpsf^ef fecti ve 
metnod tb meet -the training requirements of 
Navy jet pilots through the -year 2010. The 
VTXTS is based upon the requirement to train 
to criterion 87 Navy-provided objectives 
and any developed during the ensueing ISD 
analysis. Each of the_ contractors 1n-" 
volved in the VTX procurement was required 
to use Instructional Systems Development 
(ISD) principles and ' methodology in. the 
design of the system. The ISD analysis also 
provided answers to problems 1n areas of 
design-to-cost and life cycle cost and re- 
source requirements. Through the use of ISD, 
a usable, integrated system was created Into 
which the family of training devices were 
fully Interfaced. This paper describes the 
M down-1n-the-trenches" procedures that were 
used by the team of analysts who developed 
the functional specifications for the VTXTS 
simulator Instructor Operating Station. 
Figure 1 depicts the Mathetlcs analytical 
process which lead'to the functional spec- 
ifications fox the VTXTS Instructor 
Stations. The right hand blocks depict the 
groups or personnel who provided data to or 
received data 'from the study. The left hand 
blocks are the analytical steps used during 
the effort. 

Within the ISD process, considerable work 
was performed as part of the process of 
developing functional * specifications for the 
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Figure 1 

VTXTS Simulator Front-End Analysis 



family of simulators, 
volved were: 



The ISD steps 1n- 



. Navy Jet Pilot Warfare Specialty 

Job Analysis 
. Navy Jet Pilot Task Analysis 
. Objective Hierarchy Development 
. Media Selection 
. Syllabus Specification 
. Course Map Development ' 

Of /tfe'above step$ 1n the ISD process, the 
media selection and syllabus development 
inputs formed the base on which the analysis 
began. The media selection models suggested 
three types of devices, an Instrument Flight 
Trainer (IFT), an Operational Flight Trainer 
(OFT), and an Aerial Situation Trainer 
(AST). These devices and their associated 
capabilities support a hierarchlal acqulsl^ 
tlon (simple progressing to complex) of 
flight skills, table 1 lists the types of 
training devices, the key capabilities and 
the associated pilot training stages. 
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TABLE 1 



4 

Device 

- 




Capabi J i ties 




Traininn Staoes 


IFT 


• 


No visual 


• 


Flight Support * 


• 


riattorm notion 


• 


FAM 

V t\\\ 




• 


Flight & Ground Modes 


• 


BI 




• 


AllY-{ 1 A ZkVM TfK 

muai nary iuo 


* 


RI 




• 


CPT mode 


• 




on 




Large field-of-view 


• 


FAM 




G-suit, G-seat 


• 


D T 

BI 






Interactive Target 




. DT 






CGI Visual 


• 


All A W 

ANAV / 






• 










• 


NFORM 








• 


GUNS 




» 




• 


WEPS 








• 


TACNAV 




( 




• 




AST 


\ — 


Dynamic Visual 


• 


FORM 


• 


Dome Visual 


• 


NFORM 




• 


G-seat/G-suit 


• 


TACF 






buffet systems 


• 


GUNS 




• 


Interactive Target 


• 


WEPS 




• 


Auxiliary IOS 


• 


ACM 



Building on the data produced during the ISD 
development process, Mathetics, Inc. per- 
formed an analysis which led to a functional 
specification for the instructor operating 
stations for these devices. The design was 
to be specifically oriented toward making 
the instructor a teacher rather than an 
operator. 

^CONSIDERATIONS 

Initially MATHETICS' study team reviewed the 
literature on prior simulator design and 
applicable human factors considerations. 
Although many studies and authors con- 
tributed to the data base for this study, 
two works were particularly valuable The 
first, "The Instructor Pilot's Role in Simu- 
lation Training" by Dr. John P. Charles 
(1978), formed the basis on which the design 
algorithms were formed. The second, 
"Training Device Design: Human Factors 
Requirements in the Technical Approach , by 
Dr. Alfred F. Smode (1972), provided a 
wealth of information pertaining to design 
considerations arid presents many pertinent 
examples of human factors considerations for 
design of simulators. 

This concept of a generic instructor station 
for all of the training devices was a 
• desirable goal for the VTXTS for several 
reasons, including cost and risk reduction, 
Commonality of hardware and software, and a 



reduction in instructor training require- 
ments. The instructor consoles, although 
qenerally common, would. have some dif- 
ferences due' to the varying capabilities of 
the devices and therefore, the magnitude of 
the requirement for instructor monitoring 
and instructional activities. 



The study of all factor 
designing a generic h 



Drs to 'be considered in 
IOS revealed that the 
instructor console should have a number of 
computer-assistance features designed to aid 
the instructor. . .These features were; 



Standardization of Training 
Preprogrammed Tessons which would 
provide a series of standardized 
exercises graduated in difficulty 
with some capability for instructor 
intervention in prescribed ways to 
control the training process. This 
concept includes the possible manual 
override of defined mission events to 
accommodate students who 'perform 
below expectation for a given exer- 
cise and for insertion of additional 
rel ated events in an exerci se to 
challenge student who exceed scenario 
requi rements. These additions and 
deletions of training eyents are only 
allowed if the preprogrammed mission 
scenario software" al 1 ows manual 
intervention. 
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• Computer Assisted Measurement System 
Automated evaluation and scoring with 
objective criteria adjusted to the 
s.tage of instruction and level of 
difficulty for each student. The. 
system would provide error indica- 
tions and information which are di*r 
played at the instructor console. 

# Automated Monitor and Control 
Capability 

Computer driven multi-format, inter- 
active CRT displays at the IOS which 
present performance/error and sylla- 
bus status information i*n all 
relevant modes and in variable for- 
mats (both alphanumeric and graphic), 

* Records of Student Performance 
Software recording of student perfor- 
mance for debrief. Student perfor- 

. mance information (syllabus event 
would be error, and summary informa- 
tion) available on demand for use 
during and after* a training exercise 
(for both debrief and for record- 
keeping purposes). 

The first of these computer assisted fea- 
tures became a driving factor in the IOS 
design. The nature of the Navy Jet Training 
C^jimand requires that specific prerequisite 
knowledges and/or skills must be acquired 
prior to the follow-on event. Therefore, 
the training problem in the Undergraduate" 
Pilot Training (IfPT) context is best placed 
under software. control where the instructor 
can only minimally alter the training 
scenario. This allows for training syllabus 
configuration control across the entire Jet 
Training Command, in that the learning 
objectives, conditions and standards 
assigned to a particular lesson cannot be 
modified by the instructor who thinks he has 
"a better way 1 '. These preprogrammed lessons 
are designed in such a way as to specify the 
applicable instructional features, controls,' 
and strategies, which may be employed to 
teach certain objectives maneuvers, or 
fl ight. elements. One might contend that 
such an approach would take the instructor 
"out of the. loop" however, it is believed 
that such an approach would actually enhance 
the instructor's capability to teach rather 
than operate the device. 

Heavy programming of lessons' is also neces- 
sary where automated performance measurement 
is to be utilized. One cannot alter the 
training scenario and/or lesson methodology 
without impacting the results from the per- 
formance measurement system. Preprogrammed 
lessons also create a predictably capable 
pilot with the skills necessary for entry 
into the weapons system oriented aircraft 
located at the various Fleet Readiness 
Squadrons (FRSK Therefore, a conscious ' 



effort has been made, in .concert with ISO 
* methodology, to design out the flexibility 
of past simulators. Designing out flex- 
ibility means, designing in standardization. 
It is believed that designing out flexibil- 
ity would also reduce cost and increase 
operating simplicity. The MATHETICS study 
team held that it 1s possible to design an 
IOS which has no training controls other 
than interactive CRTs which were controlled 
by a package of training software designed 
specifically to enhance the instructor's 
capability to trauT 

The last consideration of note was the con- 
cept of separating the simulation system 
software (aero model, motion model , etc) 
from the package of training software 
(syllabus, scenario control, etc) 1n terms 
of configuration control and ^management. 
Detailed unique training software programs 
can be created for each-lesson containing 
the initial conditions .for each segment, 
allowable instructional features and strate- 
gies, and performance measurement routines. 
These packages of training software would 
act as executive routines for the entire 
training device computational system, 
prioritizing programs, routines and features 
.in terms of training requirements. The 
training software would" be under the control 
of a Training Support Center arid would be 
easily updatable as part of the training 
system quality control effort. Instruc- 
tional personnel would create the training 
methodologies which would then be imple- 
mented in software. This concept would 
allow for training implementation of the 
syllabus by flight instructors trained to 
instruct in the visual/motor skills pf 
flying high performance jet aircraft. 



ANALYSIS PROCEDURES 

Based upon the results of the initial data 
collection, the MATHETICS study team 
selected the Operational Flight Trainer 
(OFT) as the first trainer for analysis. 
This was done primarily because the OFT 
syllabus encompassed every training module 
within the VTXTS curriculum and varied in 
complexity from simple hands-on system 
familiarization to weapons delivery. The 
functional specification for the OFT could 
then be later adapted for use in the Instru- 
ment Flight Trainer (IFT) and the* Aerial 
Situation Trainer (AST), The MATHETICS 
study team selected sixteen lessons from the 
OFT syllabus for full development-and analy-* 
sis. The lessons were written from the 
existing lesson Specifications deve16ped in 
previous project work. The "sixteen lessons 
were selected for their instructional con- 
tent, the nature and intensity of instructor 
involvement; and the possibility for use of 
training features or strategies. 



Each of these sixteen lessons .was then fully 
analyzed 1n order to determine the sequence 
and validity of the learning objectives, 
detail instructor and student activities, 
and estimate the scope, possible application 
of simulator learning strategies and the use 
pf instructional features. After full lesson 
development the analysts looked at each of 
the lessons tor a number of- attributes and 
selected six of the original sixteen lesson 
scenarios for further analysis. The scope 
of the general training requirements 
included: 

• Basic Instrument Procedures 

• Procedural Demonstration 

Requirements 

• Moduli zed Lessons 

• Emergency Procedures 

• Verbally Described Demonstrations 

• self-Training Requirements 
VHigh Density Student/Instructor/ 

System Interactions 

• Visual Task Training ^ 

• Replay requirements 

• Performance Measurement 

System Requirements 

• Backward Chaining Requirements 
- * Intense Instructional Activity 

• Subjective Instructor Evaluation 

The second portion of the analysis utilized 
a functional flow analysis procedure adapted 
from the work of both Smode and Charles. 
The functional flow analysis involved the 
development of "training algorithms" aided 
by Subject Mattey^Experts (SME). The SMEs 
were asked to examine each of the lessons 
and describe how they, as flight instruc- 
tors, would best teach each of the lessons. 
This process yielded a set of eleven 
training algorithms which were entitled: 

• Pre-Briefing 

• Lestfon Modification 

• Brijefing * 
" * Initialization 

• Stdrt 

• Demonstration 

• Single T^sk Segment 

• MultipTe Repetition Segment 

(loop segment) 

• Post-Mission 

• Debrief 

• Post-Debrief 



The analysts then created strings of these 
algorithms which corresponded to the lessons 
as written in the. earlier development 
effort. The instructional methods and in- 
structional flow for each of the six lessons 
was then validated by SMEs. These string* 
of algorithms became the functional flow 
diagram for each lesson. This step in the 



process really Involves developing, an ov^ ; 
view of the Instructor's role in the 
training system in order to consolidate 
'thinking about design philosophy and aid-in 
the further definition of design require- 
ments for ihe instructor station. 

The next step, the allocation of features, 
displays, and controls, began with a review 
of the literature to determine the types of^ 
Instructor Station capabilities which were 
currently being utilized in existing and 
near term simul ators*. .These capabilities 
fell into seven categories: 

' * Instructional Features" 
s • Displays 

• Controls 

• Console Type 

• Instructor Activity Type 

• Controlling System/Software 

• Subsystem Involvement 

A "relative time-line" allocation of simula- 
tion system capabilities was then performed, 
again utilizing heavy SME inputs. The 
"relative time-line" analysis allowed speci- 
fication of activities, instructional fea- 
tures", controls, displays, etc. for each 
activity on the functional flow diagram. An 
alphabetical character was assigned to each 
of the above listed instructor/system capa- 
bilities. Each subactivity was assigned a 
numerical identifier. The al phanumeric 
identifiers were placed under 'each func- 
tional flow diagram in "relative time-line 
fashion. This data format can be seen in 
Figure 2 which depicts a portion of a func- 
tional flow diagram containing a single task 
algorithm. This format was utilized to 
determine the instructional requirements for 
each les-son and tp provide a qualitative 
indication of frequency of use and an indi- 
cation of critical ity. Additional typthe 
functional .flow analysis permitted the 
analysts to "visualize" the .desired sequence 
of instructional activities and options 
available to the instructor while actually 
involved in a training evolution. 

SPECIFICATION OF INSTRUCTIONAL FEATURES 
AND STRATEGIES ~~" 

Based upori the data that was generated 
during the \ preceding analyses, a set or 
'instructional strategies and features for 
the simulator was developed. Instructional 
strategies aVe defined as simulator capa- 
bilities whi£h are utilized as methods of 
presenting simulator training materials. 
The instructional strategies appl icable ,to 
the VTXTS 0FT\are DEMONSTRATION, REPLAY 
PREPROGRAMMED \LESS0NS, MANEUVER/SEGMENT 
RERUN, BACKWARD, CHAINING and FREE FLIGHT. 
' These strategies lare defined below. 
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DEMONSTRATION is an OFT instructional strat- 
egy that consists of a prerecorded aircraft 
maneuver, or series of maneuvers, that pro- 
vides^ mod&l of desired performance for a 
flight event. REPLAY enables the instructor 
to replay the student's performance during 
any portion or all of the most recent fivfc 
minutes of simulated flight. PREPROGRAMMED 
LESSONS comprise 98% of all training events 
and consist of a pre-defined sequence of 
maneuvers, segments of flight, instructional 
events and allowable instructional features 
and strategies which reside in the OFT 
training software. MANEUVER/SEGMENT RERUN 
is an instructional strategy which permits . 
the instructor to return to the flight con* 
dition^ which existed at the beginning of a 
. manuever, either by flight segment designa- 
tion of by .entering any mission time within 
the five minute replay recording. BACKWARD 
, CHAINING is a strategy which presents a set 

of prerecorded points tinitial conditions) 

which allow a student to learn the last 
response (last portion) In a response or 
maneuver chain first* Learning then pro- 
ceeds "backward" up the chain until all, 
membersof that chain are acquired. .FREE 
FLIGHT lessons, comprising approximately Z% 
of the scenarios, enable the instructor to 
construct his own flight scenario from a 
menu of initial conditions, malfunctions, 
flight segments, and instructional modules. 



Instructional features are defined' as simu- 
lator capabilities which enhance the 
training pf a student by modifying the 
instructional scenario or strategies,)- The 



Figure 2 

onal Flow Diagram 

features applicable to the OFT were "deter- 
mined to be SELECTABLE DIFFICULTY LEVELS 
(SDL), DIGITAL SPEECH GENERATION, and 
PARAMETER FREEZE. SELECTABLE DIFFICULTY 
LEVELS is an OFT instructional feature that 
consists of a simultaneous programming of 
three levels of difficulty for any scenario 
and which is utilized to either challenge or 
unload a student whose performance is above 
or below the UPT student average respec- 
tively. SDL is in reality a simple "poor 
man's" adaptive training capabil i ty. 
DIGITAL SPEECH GENERATION unloads the 
instructor of the requirement to act as a 
manned interactive system (GCA controller, 
ATIS, clearance delivery ACLS messages, LSO 
control, etc.) by means of speech synthesis. 
PARAMETER FREEZE allows the instructor to 
selectively freeze a certain flight para- 
meter (airspeed, angle-of-attack,- etc.) or a. 
portion of a^flight path (glideslope, cen- 
.„terLtne,^etc.)^to^w.nhin- acceptable-per-for — 
mance measurement parameters. 

- It is felt by ro^ny simulation industry 
observers that information about the in- 
tended use to-be made of a simulator's in- 
structional features, if made available 
during the - design process, coultl be used to 
design a more* effective- vehicle for 
training. The needed information must con- 
vey to the designer the prospective simula- 
tor user's concept of how the various in- 
structional features are to be employed 
during simulator instruction. It w*s 
decided by the study personnel that the use 
of Instructional Feature Design Guides, 
adopted from Pohlirfan, Isley and Caro (1978), 
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would best accomplish this purpose. The VTX 
Instructional Feature Design Guides 
accounted for .learner characteristics and 
teaching methodologies appropriate to Under- 
graduate Pilot Training. Each of the 
instructional strategies and features was 
written up in an individual design guide 
which described the feature and how it would 
be employed by the simulator instructor to 
perform coaching, demonstration, feedback 
and instructional functions for relatively 
unskilled pilots. 

A six-element format was utilized for the 
instructional feature design guides* and 
included the following items: 

• Feature name 



• Purpose and Intended Use 

• Function Description 

• Concurrent Events 

• Feature Diagram 

Figure 3 is an -example of a Feature 
Diagram. 
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"incorporate a number of features that were 
deemed appropriate as a result of the pre- 
viously discussed analyses. These capa- 
bilities of interest are as follows: 

• F-14 Instructor Support Station (ISS) 

• Interactive CRT Touchpanels 

• Preprogrammed'Lesson * 

• Digital Speech Generation 

• Performance Measurement 

System 

t F-18 OFT 

• Display System 
■* • Interactive CRT 

• Procedures Monitor 

• Preprogrammed Missions 

Of particular interest is the simplicity of 
WV*t8-OF M0S.» In -fac- t, the VH fam tly. 



of training devices all include an IOS which 
is very much sim'ilar in hardware configura- 
tion to the-2F132. However, they are 
markedly different in software design, 
operational capability and syllabus 
implementation from the F-18 OFT. 

The analyses indicated that it would be 
desirable to include two additional sub- 
systems into the simulator design to further 
enhance its capability to meet challenges 
the training system. These* subsystems were 
an Automatic Performance Measurement System- 
(APMS) and a Remote Briefing and Debriefing 
Station. The APMS would provide a computer 
system that <would score a student s perfor- 
mance automatically, and package those scores 
'in a manner which would be useable to the 
instructor in real time and during the sub- 
sequent debVief. The APMS programs wi 1 1 
compare the student's performance with a, 
definitive criterion, normalize these scores 
in order to provide peer ratings, determine 
student errors, and suggest remedial strate- 
gies. The APM system is designed in such a 
way that performance on one leg qr segment 
■ does not affect scoring on subsequent legs 

- except that the possible u*e . Of the bDL 

- feature may be continued. 

The Remote Briefing and Debrief ing.Consol e 
~c^n$4^ts-of-~an isn1atg,d„st ation interfaced 



Figure 3 
Example of a Feature Diagram 

" • * 
SYSTEM- DEVELOPMENT , 

A survey of existing and emerging simulator 
technology was conducted in order to .incor- 
porate ide-as and technology which would 
improve tj\e instructors capability to 
teach. The two devices' which most dramati- 
cally influenced. the IOS design process were 
the F-14 Instructor Support Station ,and the 
' F-18 OFT, Device 2F132. These two devices' 



to both the simulators and the Training 
Management System. This console can be 
utilized for Simulator lesson preparation, 
pre-mission briefing, p^ost-mi ssi on de- 
briefing, student data management and 
training management functions. The intent 
of the Remote Briefing and Debriefing Capa- 
bility is two fold; one, to allow briefs and 
debriefs to be held in an area, more com- 
patible to the instructional, activities 
involved in pre- and post-mission training 
than the simulator itself, and two, to 
increase the available OFT training time by 
freeing the device from the briefing and de- 
briefing activities. 



The analyst team then used the functional \ 
flow diagrams to validate the IOS design, 
ensuring that. the six lessons could be 
taught as envisioned. This validation pro- 
cess was iterative 1n nature as J^e IOS 
desfgn and the functional flow diagrams wete 
continually changed and .updated until a 
working design was completed. This analyti- 
cal, step is similar to, design "mock-up" • 
review and ensures that the instructor can 
instruct from the instructor station. The 
final step was documentation, of the actual 
fupctiorfal specifications. 

LESSONS LEARNED 

The front-end analysis leading to the VTXTS 
IOS functional specifications was not 
withtfut pitfalls. AS the analysis pro- 
ceeded, it- was necessary to workaround a 
number of problems. The following para- " ♦ 
graphs describe the most important lessons 
learned'from the Mathetics st;udy effort. 

, 1. Simulation engi neers are hardware 
. specification oriented . It was dif- • 
f leu It to communicate training require-, 
ments to the engineers dbe to the lack 
of hardware definition. Feature design 
' guides describing in detail the use of 
instructional* features to meet the- UPT . 
training requirements largety^obviated 
this problem. 

2. The JSD base on which the K)$ front-end 
, analysis is anchored must Bte completeT 
ConsideraBTe effort must be expended on 
the development of learning 'objectives, 
conditipns and standards prior to media 
selection and lesson specification. 
Improper media selection and/or poor 
lesson specifications, could lead to 
, functional specifications which do not 
address actual training requirements. 

, 3 . Subject Matter Expert (SME) participa- 
tion is essential during lesson deve- 
lopment ^ and the tuncti ona l f Tow 
analysis. WitJTout user inputsTFe 
analytical effort may end up in left 
field. SME input injects reality into 
the front-end efforts. 

4. Functional flow analyses are labor* 
mtensi ve - arid must be iterative i n 
nature . The functional flow analysi s ' 
is not the area to ski>mp on manpower. 
The work must be iterative in nature 
and previously developed algorithms 
updated as the-* diagrams are created. 

5. " The analytical "mock-up" is necessary 

to validate the f unctidnaTTn iqws . 
* Performing a "mocSf-up" review with SMEs 
identifies weak points and allowsfor, 
improvement of the specification. 
Without tt>is step, the functional flow 



diagrams, artd feature allocations will 
never communicate the instructional 
requirements. 

v 6„ Front-end analysis can yield signifi- 
cant improvement in user/manufacturer 
cooperation . SffT Inputs before the 
development ^of the functional specifi- 
cation, if properly implemented, will 
reduce tfie adverse impact of Fleet 
Project teams later. 

7. Without front-end analysis "go! d 
plating" may result ,. This front-end 
analysis showed that existing or near- 
term technology was adequate for the 
VTXTS simulators and incorporation of 
exotic features and capabilities was 
not necessary. 



FINAL COMMENTS 

A number of conferences have addressed the 
value of front-end analysis in simulator 
instructor station design in theoretical 
terms. This paper has presented an exampl*" 
of one approach to performing a front-end 
analysis leading to functional specifica- 
tions that has been tried. The methodology 
utilized in this- study is not original but 
is, adapted from previously published workr 
This front-end analysis achieved a much 
better collaboration with project engineers 
than ordinarily occurs during simulator IOS 
design efforts and thus greatly enhanced 
their understanding of the UPT training 
requirements and how ,a typical scenario 
would be taught on 'the device. The 
Mathetics study team sees the actual 
development of working specifications for a 
simulator IOS as an iterative process which 
continues throughout the simulator procure- 
men^^yeiW*- The functional specifications 
generated as a result of this front-end 
analysis was an initfal training input to 
that iterative process. The training team 
must remain involved throughout ."the 
engineering design, constructiori^and user 
acceptance processes in owder to ensure 
incorporation and proper implementation of 
key training recommendations. Without 
training requirements inputs and analysis, 
there is a risk of user dissatisfaction with 
the*device when pi *ced in the actual 
training sitiration. 
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FIGURE 1. GENERAL CONFIGURATION OF VTXTS SIMULATOR^INSTRUCTOR CONSOLE 

(US"ED ON I FT, OFT, ACMT) ' 



is located in back of and to the left of the 
student's seat, and at right angles to the in- 
strument panel .c An instructor seated at the 
.console can thus see the student and his in- 
strument panel and/or the simulated outside 
visual scen,e by looking over his right Shoul- 
der. For an ACMT, a fgll console is located 
outside the simulator, and a miniconsole con- 
& sisting of an interactive display moduTe is 
located in front of a jumpseat next to the 
cockpit. 

The configuration of the interactive mod- 
ule is presented in Figure 2. Thre switches oh 
the interactive module consist of dedicated 
special function switches and' the CRT touch 
screen, which is used in conjunction with var- 
ious types 'of menu 'displays presented on the 
CRT." The selection of membrane-type off- 
screen touch panels, over mechanical switches 
and of the^artfparent. membrane-tyj)e touch / 
screen over infrared and sonar- type touch ' 
screens was made on the basis of a detailed 
cbmpariSon'of the ease-of-use, initial cost, . 
reliability and maintainability of these al- 
ternatives. A caution in making these kinds 
of control hardware comparisons js that, the 
cost of the^ decoding and~~interface equipment 
required for different types of switches and 
switch panels is often much greater than the 
cost of the' switches Qr panels fhemselVes , 4 and 
thus must be broken out separately. 
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Functional Design 

* " The functional design of the instructor- 
console consists of several major features 
which appear capable of dramatically simplify- 
ing procedures and improving training. These 
features are as follows: 

1. -An extension of the task module con- 
cept developed, by Logicon and the Canyon 
Research Group for the F-14 OFT ISS. (Semple, 
et. al., 1979) 

2. The use of "event" modules in addition 
to task modules. 

3. The inclusion of explicit provisions 
for conducting three Basic types of simulator 
training: routine; partially specialized; and 
fully specialized. 

4. The inclusion of explicit provisions 
for controlling four basic types of simulator 
learning events: IP demonstration; autopilot 
or canned demonstration; live run; and replay. 

5. A special, minimal set of dedicated 
function switches and switch logic designed to 
interact with event modules, task modules and 
special purpose displays in a way that pernits 
highly efficient control of each type of 
training and each type of learning event. 

6. Low-density pictorial and tabular 
special-purpose displays designed to provide 
the instructor with the essential information 
at each point in the training process.- 

The following sections describe the na- 
ture, use, and projected advantages of each of 
the abov.e instructor console design features. 

Tas k Modules . As originally conceived by 
logicon and the .Canyon Research Group for the 
F-14 OFT ISS, a task module is a detaile.d pre- 
specification of the initial conditions, per- 
formance conditions, and automate^ performance 
measurement algorTttflos appropriate to a given 
maneuver or task, whity is stored in computer 
memory. One of the primary functions of such 
modules is to automate maneuver set-up. Using 
this approach an instructor can set up a simu- 
lator to train any given maneuver by simply 
identifying the name of the maneuver to the 
computer. The computer then sets up the simu- 
lator automatically according to the speci- 
fications in* the task- module. Task -modules 
thus greatly- reduce instructor workload by 
eliminating the need to remember or look up 
and then manually enter all of the information 
which must be specif ied -to set up a simulator 
to train a.specific maneuver. 

The proposed approach expands the original 
task module concept in two ways. First, the 
• task module specification is expanded to in- 
clude additional information such as 1) the^ 



methoa to be used to control the second air- 
craft in ACM and formation training; 2) the 
standard and optional instructor console dis- 
plays to be shown to* the instructor in each 
part of each maneuver; and 3) the displays to 
be recorded on videotape for later use in de- 
briefing. Second, "auxiliary" task modules 
are created which enable the instructor to 
quickly set up standard variations on basic 
maneuvers in order to tailor training to the 
needs of individual students - i.e., to con- 
duct partially specialized training as dis- 
cussed below. This is done with a minimum 
impact on computer memory requirements or 
software development costs by using auxiliary 
modules to specify only those parameter values 
which are different from those in the "core 
module for the basic maneuver. Standard var- 
iations on a given maneuver can then be set up 
by selecting a desired maneuver variation from 
a menu of standard variations. In addition to 
greatly reducing instructor workload, core and 
auxiliary task modules will increase training 
standardization to a level not possible with 
current consoles, in that all maneuvers and 
maneuver variations will be set up, displayed 
to the instructor, evaluated, and recorded in 
the same way for each student and instructor. 

The procedures for fully specialized 
training are simplified by requiring the in- 
structor to manually input only those parame- 
ter values which are different from those in 
the core or auxiliary task module under con- 
• sideration, as discussed below. 

Event Modules . Event modules are an ex- 
tension of the canned mission mode defined for 
the F-14 OFT ISS. As defined here, an event 
module is a detailed prespecif ication of the 
lesson pl^n for a given training event which 
is stored in computer memory. An event module 
is created for each simulator event in the 
flight syllabus. The event module for a given 
event specifies the specific maneuvers to be 
trained in the event, as well as the sequence 
in which they will be trained. It also spec- 
ifies the type and number of demonstrations to 
be used in the training of each maneuver, as 
well as the minimum, standard, and maximum 
number of student practice trials to be flown. 
The use of a prespecified m>ojiTj^n» standard, 
and maximum number of trials will be further 
discussed below under controls and displays. 
In general the event module will automatically 
set up the simulator to fly each run of each 
maneuver in the sequence called for by the 
lesson'plan for that event. It will do this 
by calling up appropriate task modules. For 
example, *at the start of a training session 
the event module will automatically set up the 
simulator for the first run of the first rfta- 
neuvefr^This might be, e.g., an automated 
demonstration of a la 2 y 8. After he finishes 
talking to the studejit and verifies that the 
student is ready, the instructor starts the 
run by pushing the START switch. The run will 
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ABSTRACT 

A conceptual design for a comprehensive 
instructional support system (ISS) for fighter 
and trainer aircraft flight simulators was de- 
veloped as part of an independent research 
project to develop and evaluate advanced pilot 
training techniques. The major elements of 
this ISS are: 

1. A low-workload flight simulator in- 
structor console based in part on the task 
module approach developed by Logicon and the 
Canyon Research Group for the F-14 Operational. 
Flight Trainer ISS; 

2. A comprehensive automated performance 
monitoring system based on techniques devel- 
oped by Northrop, Vought, and Systems Technol- 
ogy, Inc. (STI) 

3. A set of automated and nonautomated 

% simulator instructional features selected on 
the basis of a review of the literature and 
recent field experience 

4. A low-cost, low-workload brief/debrief 
console designed to minimize the time spent in 
the simulator oh tasks other than active 
instruction. 

These elements are designed to work together 
to reduce instructor workload and increase the 
effectiveness and efficiency of simulator 
training. Although the design is oriented to- 
wards large, highly-structured training sys- 
tems, many of its features appear applicable 
to other training environments. ■ 

INTRODUCTION 

A conceptual design for a comprehensive 
instructional support system (ISS) for fighter 
and trainer aircraft flight simulators was de- 
veloped as part of an independent research and 
development project to develop and evaluate 
advanced pilot training techniques. The major 
elements of the ISS are 1) a low-workload in- 
structor console based in part on the task 
module approach developed by Logicon and the 
Canyon Research Group for the F-14 operational 
flight trainer ISS; 2) a comprehensive auto- 
mated performance monitoring system based on 
techniques developed by Northrop for evaluat- 
' ing student performance in introductory air- 
to-air tactics, by Vought for evaluating stu- 
dent performance in carrier landing and by STI 
for examining pilot control techniques in a 
variety of tasks; 3) a set of nonautomated and 
automated simulator instructional features se- 
lected on the basis of a review of the recent 
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literature on instructional features, supple- 
mented by interviews with research and user 
personnel; 4) a low-cost, low-workload brief/ 
debrief console designed to reduce time spent 
in the simulator on tasks other than active 
instruction. 

Working together these elements appear ca- 
pable of dramatically reducing instructor 
workload and significantly increasing the ef- 
fectiveness and efficiency of simulator train- 
ing. In general, it appears that the proposed 
approach should 

1. reduce the instructors perceptual and 
procedural workload far below that of current 
simulators; 

2. reduce the time required to achieve a 
given level of student performance; 

3. permit a much higher level of training 
standardization and control than is possible 
with current simulators; ( 

4. provide a full capability for tailor- 
ing training to the needs of individual - 
students; 

5. reduce instructor training require- 
ments ; t and 

6. minimize the need to refer to user's 
manuals. 

DESIGN GUIDELINES - 

Two basic design guidelines wefe estab- 
lished for this development. The first, a 
point often stressed by*Mr. William Harris of 
the Analysis and Design Branch at NTEC, is 
that the design of all equipment to be used by 
military flight instructors should reflect the 
fact that they are typically undermanned and 
already working long hours. Their time, as 
well as tfreir energy and patience, should 
therefore be treated as a scarce, essential 
resource to be conserved wherever possible. 
It should be recognized for example that in- 
structors do not have time to stop in the mid- 
dle of a training session to try to remember a 
complex procedure or a set of parameter values 
in order to set up a given maneuver. They 
have even less time t to look up the correct 
procedure or information in a user's manual. 
Instructor consoles and other equipment to be 
used by instructors should thus be as simple, 
self-explanatory, and ea,sy to use as possible. 
Achieving such simplicity not only reduces the *. 
time normally spent in remembering, looking up 
and performing procedures, but also reduces 
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set-up errors which are time consuming and 
frustrating in themselves. Finally, -equipment 
that is simple to operate should significantly 
reduce instructor training requirements, or at 
least allow more time to be spent on teaching 
instructors how to instruct rather than how to 
operate the equipment. 
« 

The problem with all this is that making 
operating procedures simple and self-evident 
for something as complex as a flight simulator 
typically requires the development of fairly 
complex software to partially automate the 
procedures. "Rfie development of such software 
is both time-consuming and expensive. Al- 
though it seems clear that simpler procedures 
can contribute to reduced life cycle costs in 
several ways, there is simply not enough data 
at this point to prove that the projected sav- 
ings will be enough to offset the cost of 
developing and maintaining the additional 
software. A second and very important guide- 
line established for the study was thus the 
working assumption that the cost of developing 
and maintaining the software required to sim- 
plify operator procedures will be offset in 
the long run by savings in investment and op- . 
erating costs resulting from such benefits as 
reduced instructor training requirements and 
more effective and efficient student training. 
For example, the proposed instructor console 
design" appears capable of reducing the time 
now spent in deciding what maneuvers to train, 
in obtaining needed information, and in set- 
ting up the maneuvers. These savings should 
translate into reduced time in the simulator 
for each student. This in turn should resultr 
in a reduction in the total number of simula- 
tors required, and/or a reduction in the op- 
erating arid maintenance costs per simulator. 

An example of the problems caused by not 
simplifying operator procedures is provided by 
some current military instructor consoles that 
are reported to require the instructor to per- 
form as many as 40 procedural steps to set up 
a single maneuver for training. Obviously 
such a design will waste a great deal of stu- 
dent time, instructor time, and device time in 
each of thousands of training sessions through- 
out its life cycle. Lengthy set-up procedures 
lower simijlator training efficiency by reduc- 
ing the ratio of productive training time to 
total simulator time. They further lower 
efficiency and effectiveness by introducing 
continual delays in training which have an un- 
avoidable effect on student* and instructor 
concentration and motivation. 

INSTRUCTOR CONSOLE 

m Design Approach 

A high level of partial automation is used 
to achieve a very low level of workload for 
the most frequent case of routine or standards 
ized training. A moderate level of automation 
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is used to achieve a relatively low level of - 
-workload for the less frequent case of parti- 
ally specialized training. A low level of 
automation is used for the relatively rare 
case of fully specialized training, resulting 
in a level of workload for this type of train- 
ing which approaches that found on current 
consoles. 

The application of automation to the sim- 
plification of console procedures is thus in 
accordance with a frequency-of-use approach in 
that the size of the development effort. to 
simplify procedures is proportional to how of- 
ten the procedures are used. This results in 
a console that is "managed by exception" /in 
that the instructor has to do very little un- 
less he wishes to deviate from the standard 
"default" condition. From a different stand- 
point, the console design makes use of the 
principle of least effort in a way that should 
result in a much higher level of training stan- 
dardization than is realized with current sim- 
ulators. For example, the console design 
makes it easiest of all for the instructor to 
conduct the standard training specified in the 
syllabus and lesson plan; harder but still 
relatively convenient to conduct partially * 
specialized training-, and least convenient of 
all to conduct fully specialized training. 
Thus, the more the instructor deviates from 
the syllabus or leston plan, the harder he has 
to work. In this way the design strongly en- 
courages standardized training but makes it 
possible for the instructor to introduce spe- 
cialized training whenever he decides that it 
is necessary. 

Physical Configuration 

The general physical configuration of the 
instructor console consists of an interactive 
control/display module, a secondary display 
module, and a two-axis control ler^ as shown in 
Figure 1. The interactive module consists of 
a 21" calligraphic CRT with a transparent 
membrane-type on-screen touch panel or "touch 
screen", and several off-screen membrane-type 
illuminated touch switches, where individual 
switches are created from a continuous matrix 
of switches by making cutouts where desired in 
a thick, e.g., 1/8", plastic overlay. The 
secondary display module consists of a second 
21" calligraphic CRT plus a smaller CRT in- 
cluded primarily for the purpose of presenting 
the instructor with the view seen on and 
through the students HUD. 

The configuration shown in Figure 1 was 
designed for use with an instrument flight 
trainer (IFT), operational flight trainer 
(OFT), or air combat maneuvering trainer 
(ACMT). The interactive display module by it- 
self can be used as the instructor console for 
a cockpit procedures trainer (CPT), whortre it 
can be swiveled to face the student for self- 
instruction. For an I FT or OFT, the console 



stop when the instructor pushes" the STOP 
switch or when the run time reaches some pre- 
specified maximum stored in the task module 
for that maneuver. When the fun stops, the 
event module will start setting the simulator 
for the next run to be flowr>, typically a stu- 
dent practice run ("live run") for the same 
♦ maneuver. After the standard number of stu- 
dent practice runs have been flown for the 
first maneuver, the event module will start 
setting up the simulator to fly the first run 
of the second maneuver. This procedure is re- 
peated until all maneuvers have been flown. 
Unless he wants to deviate from the standard 
lesson plan, the only inputs the instructor 
has to make are to push POSITION SIM to ini- 
tialize each run, START to start each run, and 
NEXT MANEUVER to proceed to the next maneuver. 

Although it may appear at first that the 
event modules are too restrictive, a variety 
of ways are provided to permit the instructor 
to deviate from the standard lesson plan to 
tailor training to the needs of the individual 
student. As with task modules, the difficulty 
of deviating from the standard lesson plan for 
an event is inversely proportional to the es-" 
timated frequency of the need for such de- 
viation. Thus, modifying the number of trials 
within the prespecified minimum and maximum 
number of trials is very easy. Modifying the 
lesson plan to provide training on a maneuver 
which is not even in the.Jesson plan is rela- 
tively complex, although still fairly 
straightforward. 

On the surface it may seem desirable to 
let a student practice each maneuver or task 
until he reaches the specified* or customary 
level of performance for that point in train- 
ing, regardless of the number of trials called 
for in the syllabus or lesson plan. In prac- 
tice, however, most large-scale training pro- 
grams permit only slight deviations from the 
planned number of trials for each maneuver. 
This is due to the fact that large deviations 
have a chain reaction effect on subsequent 
syllabus events. For example, excessive time 
spent on one maneuver will usually result in 
insufficient practice on one or more of the 
other maneuvers scheduled for the same event. 
Jhe student is then faced with the problem of 
going to the next regularly scheduled simu- 
lator or aircraft training event without being 
able to perform these other maneuvers at the 
expected level. This is yet another example 
of how individualized progression, although 
desirable from a theoretical standpoint, 
creates serious problems in large scale train- 
ing systems. Recent advancements in dynamic 
scheduling techniques may someday provide an 
answer to this dilemma. 

Provisions for Routine vs Specialized 
Training . The proposed design makes explicit 
provisions for three types of training; rou- 
tine training; partially specialized training; 
and fully-specialized training. 



In the context of individual maneuver 
training, routine trafning consists of train- 
ing the student to perform one of the standard 
versions of the maneuver called for in the 
flight syllabus. Partially specialized train- 
ing in this context refers to tailoring train- 
ing to the needs of the student by having the 
student fly a relatively common variation of 
one of the standard versions of the maneuver, 
e..g., under different visibility conditions or 
with a different e.g. Fully specialized 
training refers to training the student to'fly' 
•uncommon variations on one .of the standard 
versions of the maneuver, e.g., in turbulence 
with one engine out, etc. 



In the context of the training of an en- 
tire event, routine training refers to con- 
ducting the training exactly as specified in a 
pre-established lesson plan, in terms of the 
maneuvers trained, the sequence of maneuvers 
trained, the type of runs or "learning events" 
used to train each maneuver and the number of 
runs of each type flown. Partially special- 
ized training here refers to making relatively 
slight variations in the lesson plan, e.g., 
varying the number of student practice runs 
for a particular maneuver within pre- 
established minimum and maximum limits. Fully 
specialized training here refers to making ma- 
jor changes in the lesson plan such as exceed- 
ing the maximum number of runs specified for a 
given maneuver or introducing a maneuver which 
is not in the lesson plan. 



As discussed above, the console is de- 
signed so that routine training, which should 
be completely adequate most of the time, re- 
quires an absolute minimum of instructor in- 
puts. Partially specialized training, needed 
occasionally, requires a moderate number of 
instructor inputs. Fully specialized train- 
ing, needed only on rare occasions, requires a 
relatively hjgh number of manual iBputs. In • 
comparison with current consoles instructor 
workload should thus be very low. for routine 
training, higher but still relatively low for 
partially specialized training, and approach 
that of current Consoles only for the rela- 
tively .infrequent case of fully specialized 
training. > • 

Provisions are also made to record the 
number and nature of each partially or fully 
specialized maneuver or task used by the in- 
structor to identify: 1) standard variations 
of maneuvers and tasks which are used ffe- 
qiejitly enough that they should be added to 
the *syl labus-to- become part of standardized 
training; and 2) nonstandard variations of ma- 
neuvers and tasks which are used frequently 
enough that they should be added to the menus 
of standard variations available to the in- 
structor for conducting partially specialized 
training. 



Provisions for Different Run Types . The 
proposed console design also makes expl i c i t 
provisions for four basic types of runs or 
"learning events" which are used in simulator 
training: IP demonstration runs; automated 
demonstration runs, student practice runs; and 
replay runs. 

In IP demonstration runs the instructor 
pilot flies the simulator from the instructor 
console usin$ the inside-out and outside-in 
graphic^displays and the two-axis controller. 
This feature was included to permit the in- 1 
structor to actively fly the simulated air- 
craft in order to illustrate-facets of perfor- 
mance that are not adequately illustrated by 
any of the automated demonstrations. Auto- 
mated demonstration runs are runs flown by an 
interactive autopilot or by a "canned" tape. 
Studtent practice runs ("live runs") and replay 
runs are self-explanatory. The use of the in- 
structor console to set up and. control each of 
these types of runs is described in the fol- 
lowing sections. 

Controls . Figure 2, above illustrates the 
three basic types of controls provided on the 
instructor control/display module. These are: 

1. A transparent CRT touch screen used to 
•select items from alphanumeric or pictorial 

menus displayed on the CRT. 

2. Dedicated function switches located 
both above and below the CRT. These are 
groupecL into five subpanels corresponding to 
five basic function? performed by the instruc- 
tor: simulator status mOnitoriRg; event con- 
trol; maneuver or task control; number entry; 
and malfunction insertion. The switches used 
to perform each of these functions are dis- 
cussed below. * 

3. A two-axis controller located to the, 
right of the control switch slant panel. (For 
use with an I FT, OFT or ACMT. Not required < 
for a CPT.) The twcf-axis controller can be 
used by the instru/tor a) to fly the student's 
aircraft during manual demonstrations of cor- 
rect maneuver performance ("IP demos"); b) to 
manually control the simulated enemy aircraft 
during ACM and gunnery training; and c) to fly 
one of the friendly aircraft during close for- 
mation or traffic pattern training. 

Five types of dedicated function indicator 
switches were defined for the console: 1) 



those used to control and monitor the status . 
of the simulator itself; 2) those used to 
identify and modify the event to be trained; 

3) those used to monitor and control the 
training of individual maneuvers and tasks; 

4) *those used to enter numbers into the com- 
puter (e.g. to change the value of parameters 
defining initial conditions, etc.); and 5) 
those used to insert, monitor, and remove 
malfunctions in the simulated aircraft. The 
subpanels containing each of these types of 
switches are shown bejow in Figures 3 and 4. 

The Simulator Status Panel, the Numeric 
Entry Panel, and the Malfunction Select Panel 
are self-explanatory. The functions asso- 
ciated with the Event Control Panel and the 
Maneuver Control Panel are described below. 

Event Control Panel . The Event Control 
Panel is used as fol lows. Pressing the STAGE * 
switch causes a menu of the different stages 
of training to appear on the CRT display, 
e.g., Instruments Formation, ACM, etc. In- 
dicating a specific stage with the touch 
screen calls up a menu of the different train- 
ing events (simulator sorties) which must be 
flown during that stage. Indicating a spe- 
cific event on this menu causes the detailed 
lesson plan forthat event to be displayed, 
showing the sequence of maneuvers and tasks to 
be trained, and the minimum, standard, and 
maximum number of irials for each maneuver or 
task. If the lesson plan is more than one 
page long, pressing a point on the bottom 
right (or left) corner of the touch screen 
will cycle forward'(or backward) through the 
pages in the menu. Indicating a specific 
event $lso causes the event module^for that 
event to be loaded into main memory. Pressing 
the START EVENT switch then calls up the task 
module for the first maneuver or task to be 
trained and starts setting up the simulator 

'for the first trial of the first maneuver or 
task. (Where desirable, this could^be mech- 
anized so that an instructor could start an 
event from somewhere other than the beginning 
by simply touching the screen to indicate the 

'first maneuver or task to be trained befor^ 
pressing the START EVENT switch.) 

' The LESSON PLAN switch shown in Figure 4 
can be used to call up the lesson plan for re- 
view at any time during the event. The lesson 
plan can also be used as a menu to conduct 
specialized training by , introducing maneuvers 
and tasks out of sequence, e.g., to skip a 
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section of the plan, or to return to a maneu- 
ver already completed. Similarly, event menus 
and stage menus can be used to introduce maneu- 
vers normally practiced only in other events 
or even maneuvers normally practiced only in 
other stages. Again, the more radical the 
departure from the standardized training pre- 
specified for the event, the greater the. num- 
ber of procedural steps required. 

Maneuver Control Panel . The Maneuver Con- 
trol "TaneT^TTljsed~ar7o^ows . The POSITION 
SIM, START, FREEZE, and STOP switches were 
carefully selected as a minimal, necessary and 
sufficient set of_simulator run controls on 



the basis of our experience in conducting sim- 
ulator training experiments over the past sev- 
eral years* The function of these switches is 
described below in Figure 9 which presents a 
detailed sequential description of instructor 
control inputs and display indications for a 
standard prerecorded maneuver demonstration. 
These controls function in exactly the same 
way for an instructor flown demonstration (IP 
DEMO), autopilot or prerecorded demonstration 
(AUTO DEMO), student practice run (LIVE RjJN) , 
or replay. The functions of the NEXT MANEUVER" 
and EXTRA TRIAL switches are described below 
in the discussion of the Quantitative Perfor- 
mance Display. 
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Displays . Dynamic "inside-out," and 
"outside-in" pictorial representations of the 
student's aircraft and its relation to other 
aircraft or ground features are now relatively 
common, at least for certain maneuvers. For 
example, many current instructor consoles can 
show dynamic two-dimensional pictorial 
"representations of aircraft glideslope and 
localizer error. Similarly, dynamic three- 
dimensional perspective outside-in and inside- 
out pictorial representations of ACM 'engage- 
ment like those developed by Cubic for the* 
Tactical Air Combat Training System/Air Combat 
Maneuvering Range ( TACTS/ ACMR) have been de- 
veloped for ACM simulator instructor consoles 
by Northrop and other companies. (Spring, - 
1976) In the proposed design,' dynamic inside- 
out and outside-in views of the maneuver being 
floy/n are combined With a number of special 
purpose displays to provide the instructor 
with the essential information required at 
each point in training. Figure 5 shows an 
outside-in display of an ACM engagement. The 
numbers in Figure 5 show the relative position 
of attacker and bogey at four different times 
during the engagement. 




FIGURE 5. OUTSIDE-IN DISPLAY OF ACM ENGAGEMENT 

As noted previously, the generalized (all 
except CPT) configuration of the instructor 
console has three CRT displays. These are: 

1. A small CRT at the top of the left- 
hand or "secondary display" module. This is 
normally used as a repeater display to show 
the instructor the view through the HUD and 
the HUD symbol ogy being seen by the student. 

2. A 21" CRT on the secondary display 
module. During a run this is normally used to 
present a dynamic outside-in pictorial rep- 
resentation of the student's aircraft and its 
relation to other aircraft or features on the 
ground. 



3. A 21" CRT on the interactive control/ 
display module. During a run this is normally 
used to present a dynamic inside-out pictorial 
representation of the view through the wind- 
screen of the student's aircraft. Between 
runs this CRT is used to present special- 
purpose displays which provide the instructor 
with a variety of additional information 
required to set up, conduct, monitor, and 
evaluate training. 

A description of the primary special - 
purpose displays developed during the design 
study is presented in the following sections. 

Maneuver Description Display . A sampl£* 
Maneuver Description Display is shown in Fig- 
ure 6. This display is automatically pre- 
sented 'to the instructor at the beginning of a 
training for each new maneuver or task- The 
■display shows: * 

1. The type and number of demonstrations 
to be flow; and the minimum, standard and max- 
imum p-'ifcer of student practice or "live" runs 
to be t } own following the demonstrations. 

2. The standard or "desired" time of oc- 
currence of critical points or "windows" which 
normally occur during the maneuver, on a time 
scale appropriate to the maneuver. 

3. A detailed description of the perfor- 
mance standards, initial conditions, and per- 
formance, conditions for the maneuver to be 

f 1 own . 

The purpose of the maneuver description 
display is to refresh the instructor's memory 
as to a) the way in which the maneuver should 
be flown and b) the specific criteria which 
should be used to evaluate student 
performance. 

Quanti tati ve Performance Pi spl ay . A typi- 
cal Summary Quantitative Performance Display 
is shown in Figure 7. This display is 
automatically presented to the instructor at 
the end of each triaj. The display shows: 

1. An indication of the training status 
for the maneuver. This part of the display is 
unchanged from that shown in the Maneuver De- 
scription Display except that the number of 
the trial just completed is brightened to show 
the instructor the number of trials completed 
and the number yet to be trained. * 

2. Both the actual and desired (standard) 
time of occurrence of each critical point or 
"window" for the maneuver. This display was 
added because of analytical studies which sug- 
gest that the times of occurrence of critical 
points in a maneuver are sensitive indicators 
of overall maneuver performance. 

3. A description of the most important 
out-of -tolerance conditions detected at each 
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window or in the segment or "corridor" irmiedi- 
ately following that window. The out-of- 
tolerance conditions for windows and corridors 
are detected by the. automated performance mon- 
itoring system discussed in the following 
section. 

As illustrated in Figure 7, the Summary * 
Quantitative Performance Display shows out-of- ' 
tolerance conditions detected for each window 
and succeeding segment. Each out-of-tolerance 
condition is indicated by a very brief English- 
language description of the out-of-tolerance 
condition, followed by the observed value of 
the parameter and the standard or desired 
value in parentheses. For example, Figure 7 
shows that the pull-up occurred very late, 
that it was initiated when the attacker was 
1/2 miles from the bogey, and that is should 
have been initiated when the attacker was ap- 
proximately 7/8 of a mile from the bogey. The 
values in parentheses serve to continually re- 
mind the instructor of the qualitative.perfor- 
mance standards established for the maneuver. 

It would of course be possible to present 
this same display to the student by projecting 
it on the simulator's image presentation sys- 
tem or by displaying it on the student's mul- 
tifunction display in the cockpit. This 
feature could t>e used as a tool to assist the 
instructor in providing feedback to the stu- 
dent. With appropriate controls. added 'to the 
cockpit this could also be used to provide the 



simulator with a capability for self- 
instruction, which can be highly efficient 
when alternated with conventional instruction. 

It should be noted that the display pre- 
sents only the two or three most important 
out-of-tolerance conditions detected for each 
window and accompanying segment. This is done 
to prevent overloading the instructor with too 
much information. It is done by mealns of a 
table look-up system which identifies the most 
important errors by consulting a table con- 
taining the prespecified rank order of impor- 
tance of each of the known errors for each 
window and segment for the maneuver. This 
table is part of the automated performance 
monitoring algorithm which is in turn part of 
the task module for the maneuver. If de- 
sired, the instructor can- obtain information 
on additional performance information for any 
window and segment by simply touching" the name 
of the window. This causes a Detailed Quanti- 
tative Performance Display of all the out-of- 
tolerance conditions detected for that window 
and segment to be displayed. This detailed 
display for a given window and- segment would* 
probably 'group out-of-tolerance conditions -by 
parameter type. to make it easier for the in- 
structor to read, e.g., under headings such as 
"Energy Parameters," and "Relative Geometry * 
Parameters. 41 ' * • 

The time scale at the top of the display 
has a second, possibly very useful function. 
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FIGURE 7. SUMMARY QUANTITATIVE PERFORMANCE DISPLAY 



By pressing any two points on the scale the 
instructor can preset the starting and stop- 
ping points of any automated maneuver demon- 
stration or replay. This might be done to 
save time or to highlight some aspect of ma- 
neuver performance. Thus, in the example 
shown in Figure 7, the instructor could use 
this technique to quickly set up the simulator 
to replay a just completed live run from a 
point just before the low reversal to a point 
just after the inverted position, in order to 
.illustrate some detail of correct or incorrect 
' performance in that p§rt of the maneuver. 

A Similar capability can be provided for 
live runs by developing additional software to 
enable the computer to pick of^onstapda** 
initial conditions,. etc. from* intermediate 
points in a canned demons trartipirof the ma- 
neuver. The ability to set yp arbitrary start 
and stop points for live runrs provides an in- 
herent capability for backward chaining and 
similar instructional techniques as discussed 
below in the section on instructional 
features. 

In addHjon to ejp^ing the instructor to 
keep track djNt^^he is in the training for 
each maneuyjM* the part of the display showing 
the trials completed and trials remaining are 
used by the instructor in connection with the 
NEXT MANEUVER and EXTRA TRIAL switches shown 
above in Figure 4. 



Normally, 'as soon as the standard number 
of student practice trials ("live runs") have 
been completed for a given maneuver, the in- 
structor will debrief the student and then, 
press the NEXT MANEUVER switch. This will 
cause the simulator computer to call up the. 
task module for the next maneuver specified in 
the event module and start setting up the 
simulator for the first trial of the next 
maneuver, typically a demonstration. If the 
standard number of practice runs have not been 
completed but the instructor decides' that no 
further practice is necessary, and if the num- 
ber of trials completed is equal to or greater 
than the minimum shown on the display, the in- 
structor can move directly to the next maneu- 
ver to by pressing the NEXT MANEUVER switch. 
Similarly, if the standard number of practice 
runs have been completed but the instructor 
fes the student needs more practice, and 
if thTTmmfeec^of trials flown is Less than the 
maximum shown oJrthe-te&ti)^^ 
can set up- an extra trial on the same maneuver 
simply by pressing the EXTRA TRIAL switch. In 
this way the instructor can easily vary the 
number of trials flown within prespecified 
limits to tailor training to the student's 
needs. He cannot exceed these limits without 
going through a more complex procedure. This 
feature is designed to make it difficult to 
reduce the number of trials to the extent that 
a student receives little or no practice on a 
maneuver, or to increase the number of trials 
to the extent that insufficient time is left 
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for training on remaining maneuvers. Here* 
again, the actual number of trials flown on 
each maneuver could be recorded to provide an 
empirical basis for modifying the minimum, 
standard, and maximum number of trials spec- 
ified in the event module. % 

Other Displays. Several other displays 
were developed to reduce instructor workload 
for various tasks. For example two- 
dimensional tabular menus Were developed to 
enable the instructor to quickly set up the 
desired mode of controlling each aircraft in a 
three ship formation, or in two-on-one or one- 
on-two ACM engagements. This type of display 
is illustrated i-n Figure 8. 
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FIGURE 8. MENU 'DISPLAY FOR SELECTING METHOD 
^ ^CONTROLLING EACH AIRCRAFT FOR ACM 

DetaileyTungtions Analyses . Detailed 
functions analyses have been developed showing 
exactly how the Event Control Panel , Maneuver 
Control Panel, Maneuver Description Display, 
and Summary Quantitative Performance display 
would be used to carry out each of the follow- A 
ing instructional functions and subfunctions: 

Identify event to be trained 

Conduct standardized training 

- demonstrate maneuver 

- instruct live run 

- replay live run , 

Conduct partially standardized training 

- modify start-stop points 

• - modify number of trials per maneuver 

• normal transition 

• skip to next maneuver 

• give extra trials on current 
maneuver 0 

- introduce standard variation of 
maneuver 

Conduct fully specialized training 

- introduce nonstandard variation of 
maneuver * 

- introduce m^peuver out-of-sequence 

- introduce maneuver not contained in 
event module 



Insert and remove" malfunction 

Specify method of controlling* each aircraft 
for ACM and Formation Flights. 

A sample functions analysis for demonstrat- 
ing a maneuver is shown in Figure 9. This 
type of analysis is an important .first step in 
developing instructor console software in that 
it can fairly easily be translated into a com- 
* puter flow diagram. 

Automated performance Monitoring (APM) System 

As discussed in the previous section-, the 
automated performance monitoring (APM) system 
generates a quantitative performance display 
at the end of a maneuver which is a summary of 
•the most important out-of-tolerance conditions 
• detected during the maneuver. Art example of a 
quantitative performance display i^ shown 
above in Figure 7. This information is 0re- 4 
sented to the instructor as an aid for evalu- 
ating and diagrjo^'ng student performance on 
the maneuver. 

*» , 

The APM system is based primarily on tech- 
niques developed by -Northrop, Vought, and Sys- 
tems Technology, Jnc. (Carter 1976; Carter 
1977; Sepp 1977; Heffley et. al. 1982). It 
also uses APM concepts developed by Appli- 
cation, inc., the Canyon Research group, and 
Vreuls Research, Jnc. '(Semple, et. al. 1979; 
Vreuls, et. al. 1975) 

The system is designed to detect three 
types of errors: flight path errors, control ' 
technique errors, and procedural '(switch se 7 
quence). errors. In all, cases, errors or out-- 
of-tolerance conditions are detected by 
comparing the observed' value of a parameter 
for a specific point or segment in a maneuver 
- or task with the standard value and tolerance 
limits stored in computer memory. Flight path 
errors, e.g., "excessive airspeed at roll in," 
are currently being measured in- this way by* 
the Vought A-7 NCLT APM system now in opera- 
tional use at NAS Lemoore. Control technique 
errors, e.g.;* "controlling attitude instead of 
Nz" can be measured in this way by pilot be- 
havior identification techniques like those 
developed by STI, which were used in Northrop > 
simulation studies to evaluate ^effective- 
ness of alternate training aircraft configur- 
ations for carrier landing. Procedural" 
errors, e.g., "arming switch -thrown out of 
sequence" are now being detected in this man- 
ner by the EA-3 Low-Cost CPT developed by 
Application, Inc., and several other systems. 

The potential impact of^PM techniques on 
the effectiveness and efficiency of simulator 
training has been demonstrated by studies on 
the inter- and intra-obser'tfer reliability of 
instructor judgments. (Carter, 1976; >Knoop & 
Welde, 1973) These studies show that instruc- ' 
tor judgments of overall maneuver performance 
are. fairly reliable, but that instructor judg- 



CONDUCT STANDARDIZED^ A INING. * " 

t* Follpwing the identification of the event to be trained^the instructor conducts a standardized train** event by using the Maneuver 
Control Panel and the special purpose displays as follows: A > * > 

1 ~ Demonstrates Maneuver (AUTO DEMO) V * * 

*' a Presses "POSITION SIM" sw.tch which causes.the^POSITION SIM" ligfit <o start blinking and the computer to start setting 

• *. • " up the simulator for the prerecorded demonstration specifiedfor this maneuver in the task module. < 

" ' f b. ("POSITION SIM" switch comes on steady ajKi first frame of HUD, -flight path, and instrument repeater dynamic disp lays 

* are displayed on instructor console CRTs, indicating that simulator is ready to fly.) - - v / J" 

* % c. Comments on the demonstration to be flown and verifies that the student is ready b 

d. Presses "START" switch to start the prerecorded demonstration. 

e. Makes comments during the demonstration. * . * 

f . Presses "FREEZE" switch to freeze demonstration in mid-air. (This freezes the visual scene, motion rjlatfotfn, g-seat, 
g-suit, instruments, etc, in the position existing at time-Of freeze> % . ># ) 

- g. Presses "START" switch to restart prerecorded demonstration'from frozen position, 
h Instructor presses "STOP" switch/demonstfation reaches auto stop point.j 

{ 1 ) Turns off visual scene ^nd returns mbtion platform, g-sdat, g-suit, instruments, etc, to a neUtral position. ■ 

(2) Freezes last frame of HUD and flight path displays. * u • ror 

(3) Causes Summary Quantitative Performance Display and Grading Scale Display to appear on the mteract.ve CRT. ^ - 

• . VlGURE 9. 



SAMPLE. FUNCTIONS ANALYSIS 



ments relating to the acceptability of spe- 
cific parameters at specific points or over 
specific segments daring the maneuver are 
highly unreliable. Since these judgments are 
the basis for the instructor's diagnosis of 
the causes of substandard maneuver perfor- 
mance, it follows that much of the feedback 
currently g-iven to students is in error.' The* 
effect of this on. the effectiveness 'and'effi- 
ciency of training can be understood -ty con- • 
sidering the effect of telling a Student he 
pulled up late when he. actually pul>ed up a * 
early, whichis precisely what the data indi- 
cate is happening, \ 

i* , • \ \ 

The proposed APM^ system is designed to ■ * 
correct this proWem by* providing instructors 
with objective data which can be used to give* 
students more accurate feedback. This same 
data can be recorded to provide-an objective 
data base for improved management of student^ 
progress, improved student selection, and more 
.accurate evaluation of total system 
performance, * , 

Although the benefits of the proposed APM 
system are extensive, the-ctet of developing 
and perfecting an APM algorithm for a single 
complex maneuver can be quite high. For 
.large-scale .training systems a phased APM de- 
velopment may be required to spread these 
costs over t*im?. 0 In this approach algorithms 
are first developed , only for selected, skfl.l- 
sensitive maneuvers. Algorithms for addi- 
tional maneuvers can; then be deyeJopecKas part 
of an ongoing ISO effort. 

" instructional features " 0 

A review of "current literature on instruc- 
tional features served as a starting point for 
the identification' and evaluation of candidate 
instructional features, (e.g., Bailey &' 
Hughes',' 1980; Caro et..al- 1*979, 1980; Hughes, 
1978, 1979; Lintern &Gopher, 1977; Lintern 
1978; Semple, et. al. 1980; Shaw 1979; and 
Weller, 1979} Candidate instructional 



features v/ere classified into two broad types: 
nonautomated and automated. 
^ 43 * 

Nonautomated Features . * Figure 10 ill.us*7 
" trates the results of an" analysis performed to 
evaluate a variety of nonautomated instruc- 
tional features .identified as candidates for 
simulator .tratning., Ill general, the features 
shown as rejected were rejected because of a 
total lack of empirical data on their training 
effectiveness. . * x 

Backward chaining has been found to be a 
highly, effective simulator training technique 
in an experimental setting. (Hughes, 1979) 
It i si .therefore* recommended for IFT; OFT;, and 
. ACMT* training pending further investigations 
of its tost-effectiveness inAnilitary pilot 
training programs!; the" ins teuctor -console fea 
ture used to *et up arbitrary start and stop 
•• points for liive runs provides a basic capabil- 
. ity fpr backw^d chaining and similar instruc- 
/ tiional techniques, -as discussed above in the , 
^ectf&n on instructor cpnsole displays. 

East time has been found to be very effec- 
tive simulator training technique at NASA-Ames 
for preparing NAS£ test pilots for the severe 
time-compression effect encountered in actual 
test flights. Fast time is a technique 
■ whereby the program" integration 1 interval is 
'greater than the elapsed real-time interval, 
causing frequencies and velocities to in- 
crease, i.e., things happen more quickly than 

"in real flight. Fast time is thus a form of 
overtraining which 0 cou1d conceivably increase 
the training effectiveness of IFT and OFT 
training by bfetter preparing students for the 

'increased stress and workload Associated with 
aircraft flights. Like other types of over- * 
'training l.t could also result in excessive 
simulator tr^ining'TTme. .While current 5imu- 
* lator computers have an -inherent capability ' 
for fast time, the cost of upgrading image 

'generation computers, motion system s£rvo^; 
and other hardware to achieve the required 
/ frequency response would be very high. This 
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FIGURE 10.. . EVALUATION OF NON-AUTOMATED INSTRUCTIONAL FEATURES 




technique is therefore not recommended pending 
a' more direct demonstration of its effective- 1 , 
ness for mil itary »fl ight training. 
* • * «- 

•Landing-point predf.ctgr- displays and aug- 
mented visual cues such as the "pole track" . 
-display are -recommended as selectable OFT HUD 
% " " 'ays for field and carried landing train- 
ing^. The "pole track" is a display of succes- 
sively shorter "poles" on eadbside of the 
glideslope" showing the correct guideslope and 
providing an external indication of aircraft 
velocity. The effectiveness of such displays 
.for simulator training in field and carrier 
landings* has been demonstrated in the studies 
by Lintern (1978) and Weller (1979). • 

Electronic tracer bullets and funnel dis- 
plays like those on the G.£. HUD are recom- . 
mended'as selectable ACMT HUD displays for 
air-to-air gunnery training. These displays • 
have been shovm to be extremely effective for 



teaching a student the effect of aircraft 
control inputs on the dynamics of the bullet 
stream. A continuously computed impact point, 
velocity vector, and bomb fall line are recom- 
mended as selectable OFT HUD displays for air- 
to-ground weapon delivery training. It is 
assumed that these HUD displays would dupli- 
cate those in the aircraft. , 

Automated Features . Figure 11 "illustrates 
the results of an analysis performed to evalu- 
ate automated instructional features which 
were identified as candidates for an ACMT. 

The capability for preprogrammed initial 
conditions and preprogrammed malfunctions is 
inherent in the task module concept." Tech- 
niques for using normal and emergency proce- 
dure task modules in conjunction wi,th other 
types of task modules are described in Semple, 
et. dl. (1980). 
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FIGURE 11. EVALUATION OF AUTOMATED INSTRUCTIONAL FEATURES 



An automated ground controller similar to 
that developed for the F-14 OFT Instructional 
Support System is recommended for IFTs and 
OFTs for simulating the ground controller's 
"voice instructions in ground-controlled ap- 
proaches and carrier-controlled approaches. 
Instructor pilots consulted agreed that in- 
structor pilots cannot accurately simulate 

• ground controller voice instructions due to 
the specialized training and practice re- 
quired. This feature should thus increase the 
training effectiveness of IFTs and OFTs while 

, reducing instructor workload. 

Maneuver-specific "autopilot bogies" are 
reconmended for controlling the simulated 
bogey and wing-man in introductory ACM train- 
ing and for controlling the lead aircraft, 
second aircraft, ~dr third aircraft in close- 
and tactical -formation training. This ap- 
proach was used with success in ACM simulator 
training experiments performed by Northrop for 
NADC.ip 1976. (Spring, 1976; Carter, 1976) 
The autopilot bogey is used for introductory 
training in classic maneuvers. Typically, it 
is^a relatively simpte algorithm which causes 

* tffe bogey or friendly aircraft to react to a 



change in student aircraft position in the 
same way that the instructor would react at" 
this stage of training. For example, a barrel 
roll attack bogey is programmed to tighten its 
turn as the student reduces angle-off-rtail and 
nose-to-tail, and to reverse when the student 
overshoots. Autopilot bogies increase OFT and 
ACMT training. effectiveness in two ways: 1) 
they provide much more realistic training than 
is possible with noninteractive prerecorded 
bogies; and 2) they significantly reduce the 
need for the. simulator instructor to fly the 
bogey aircraft in ACM and formation training, 
thus permitting the instructor to concentrate 
on observing and coaching student performance. 



used 



The .term "generalized autopilot bogey ,r is 
used here to refer to 'an autopilot bogey which 
is capable of flying an entire ACM engagement' 
against a human pilot. This type of bogey is 
us^d in the Northrop Large Amplitude Simulator 

AS.), the Simulator for Air-to-Air Combat 
(SAAC) at Luke AFB, and the 2E6 ACM simulator 
at NAS Oceana. 'A generalized autopilot bogey 
is recommended for ACMTs to be used in ad- , 
vanced training but wolild probably not be 
cost-effective for undergraduate ACM training. 
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As shown in Figure 11, an automated per- 
formance monitoring system is recommended for 
all four types of simulators. As described in 
the previous section, the APM system detects 
out-of-tolerance conditions during each trial> 
of each maneuver or task, and displays this 
information to the instructor at the end of 
the trial. .By providing more detailed and. 
t^more accurate data on Student performance, the 
APM system performs three essential functions. 
First, it increases the effectiveness of ac- 
tive instruction by providing the "instructor 
with objective information which can be used 
as an aid in evaluating and diagnosing indi- 
vidual trials. Second, it puts the management 
of student progress through the syllabus on a 
more objective basis by providing instructors 
with more complete and accurate data on which 
to base grades.\ Finally, it provides a- 
greatly increased system evaluation capability 
by' providing operational ISD personnel with 
objective data on the effects of variations in 
training content, sequence, techniques, and 
equipment. 

Automated audio alerts are recommended for 
all four types of simulators £o alert the in- 
structor to certain critical student errors 
which might otherwise be missed. This enables 
the instructor to correct errors as soon as 
they occur to prevent formation of unsafe 
habits? such as pulling too many Gs. This is 
done by means of audio tones and/or digitally 
recorded voice messages. 

Automated verbal CQaching and cueing of 
the student during a maneuver is, not recom- 
mended because of research evidence that such 
Coaching is at best distracting, and at worst 
constitutes an irrelevant cue to correct ma- 
neuver performance which can be used con- 
sciously or unconsciously to "beat the 
system." 

Automated grading is recommended for field 
and carrier landing training in OFTs because 
of the demonstrated agreement between A-7 NCLT 
Landing Signal Officer (LSO) grades and the 
grades assigned by the A-7 NCLT automated 
grading system developed by Vought. 

A capability for fully automated Computer- 
Assisted Instruction (CAI) is recommended only 
for CPTs. At s the completion of each procedure 
this system provides automatic feedback to the 
student by displaying the errors and the cor- 
rect procedures oh the instruct%r7student con- 
sole CRT display. The system then instructs 
the student to reset all switches to their 
normal positipjns. Depending 6n performance to 
that point, it then instructs the student to 
perform the previous procedure again or to 
start training oh a new procedure. 

Adaptive training is recommended for OFTs 
in conjunction with an A-7 NCLT-type grading 
system to vary the difficulty of field or car- 
rier landing as a function of student perfor-' 
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mance. Task difficulty is decreased by 
displaying a predictor display' and augmented 
visual cues such as a "pole track" display on 
the student '"s HUD. Task difficulty is in- 
creased by removing these displays. While it 
is possible for .the instructor to insert and > 
delete these displays manually, the training 
effectiveness of these displays is greatest if 
they are inserted and deleted according to an 
.optimized adaptive training algorithm. 
(Lintem, 1978) 

• 

Adaptive training in which task difficulty 
is varied by varying system dynamics, (e.g., 
changing the stability of the simulated air- 
9 craft) has been shown to be ineffective and is 
not recommended. (Shaw, 1979). 



BRIEF/DEBRIEF CONSOLE 

The brief/debrief console for the proposed 
ISS is essentially a CAI terminal interfaced 
to a videodisc system, which has an added 
video cassette playback capability for de- 
briefing. It is assumed that the brief/ 
debrief console would-be interfaced to a com- 
puterized scheduling and record? keeping 
system. 

** i 
Physical Configuration 

The physical configuration of the console 
consists of a 21" CRT display with a transpar- 
ent mpmbrane-type touch screen and a set of 
dedicated function switches located on an off- 
screen membrane-type touch panel, where indi- 
vidual switches are located and grouped by 
means of cutouts in thick plastic overlay. 
Ideally, the brief/debrief console would be 
identical to the CAI/videodisc terminals used 
for academic training, with the exception of 
the video cassette unit and -additional func- 
tion^witches used only for debriefing. A 
detachable alphanumeric keyboard i^required 
to permit the IP to enter nonstandard comments 
in the students computer file. 

Functional Design ■ 

The console is designed to be used for 
both simulator and aircraft flights. It sup- 
ports five major instructional functions, as 
follows: 

l t Event Planning .* The console is used 
by instructors to determine student status; to 
review the student's performance in past 
events; to request schedule changes if rjeces- 
sary; and to identify minor changes" in the 
standard lesson plan which could benefit the 
sttide/it. 

v 2. CAI Brief W and Test . The console is 
ustoi to give students a CAI review and test on 
acaofemic and flig^support materials relevant 
to tire flight to be flown. 



3- IP Briefing , The console can provide 
a hard copy of the CAI test results to be use^d 
by the IP as a starting point for the IP brief- 
ing which follows the CAI briefing. (A hard 
copy printer could be time shared by several 
brief/debrief consoles.) 

4. Debriefing . The console can be used 
for debriefing both simulator and aircraft 
flights, assuming that the aircraft has an 
onboard vide^ recorder* 

5. Data* Entry . The console is used by 
the IP to enter grades and comments on the 
flight in the student's computer file. 

Debriefing Displays 

For simulator debriefings, a multiplexing 
scan converter is used to make a two-channel 
videotape record of what was shown during and 
after each maneuver on any. two of the instruc-^ 
t6r console CRT displays. Thus one videotape w 
channel" will typically show a dynamic "inside- 
out" presentation Cff the pilot's view through 
_^|he windscreen during the maneuver, including 
cbctinuously changing digital readouts of key 
parameters on the edges of the display. • This 
is/ followed by the Summary Quantitative Per- 
formance Display shown to the instructor at 
" the end of the maneuver, plus any detailed 
quantitative performance displays requested at , 
that time. The second channel will typically 
show a dynamic "outside-in" presentation of an 
outside observer's view of the maneuver/ also 
including digital readouts. This is followed 
by a static graphic display of the desired ancl 
actual flight path flown. The instructor can 
switchback and forth between these displays 
as ^required during debriefing. 

* Debrief Procedure v 

For debriefing, the, brief/debrief console 
is used by the IP and the student to review 
•the videotape record of the aircraft or simu- 
lator flight'* to be debriefed. Fof* both air- 
craft and simulator debriefings it is expected 
that on,ly selected segments of the tape would 
be reviewed, since reviewing the entire tape 
would require as rmjch time as the original 
flight. To make this process as efficient, as 
possible, three different methods are provided 
to enable the IP to quickly get to a point-of 
interest on the tape. First, he can slew di- 
raptly .to any electronic flags which he in- 
serted on the tape during the. event. Second, 
he can slew to a specific trial- of a specific 
maneuver, using a display of the lesson plan 
as a menu. .(This approach wijl be most effec- 
tive for simulator debriefings, due to fre- 
quent changes in lesson plans which are 
dictated -by events an^conditions in the air.) 
Third, he' can use fast-forward and rewind con- 7" 
trols to slew to an approximate time in the 
flight, where the approximate flight time at 
each point on the tape will be continuously 



displayed on the screen. Once the approximate 
point is located by one of these three meth- 
ods, the precise segment of the tape to be re- 
played will be controlled by conventional tape 
playback controls (fast-forward, rewind, stop, 
,pause, and play.) Detailed functions analyses 
"have been developed which describe the proce- 
dures used for slewing to a desired point on 
the tape with each method. 



Data Entry Procedure 

The instructor grades each event using the 
console touch screen and an event-specific 
grading form displayed-on the-CRI*— The- grad- 
ing form consists of a matrix of grading cat- 
egories listed vertically down the left side 
of the display and a grading scale displayed 
horizontally across the top. To grade a spe- 
cific category, the instructor simply presses 
the touch screen across from the category in 
the column corresponding to- the desired grade. 
Standard comments on the students performance 
are entered by -pressing the touch screen next 
to rme or more appropriate comments in a menu 
of standard comments. Nonstandard comments * 
are typed in with the alphanumeric keyboard. • 
It, is assumed that all grades and comments 
entered by the instructor at the console would 
be automatically entered into a centralized 
computer file containing a record of the stu- ' 
dent's performance in each event. 

CURRENT STATUS AND FUTURE PLANS 

An independent research and development 
project is now^in progress to develop a work- 
ing model of the Summary Quantitative Perfor- 
mance Display. Current plans are to follow up 
thjs Effort by developing working models of 
thither instructor console features described 
abdve. 
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INTRODUCTION 



The Offensive Avionics System (OAS) PPT 
(See Figure 2) includes two user interfaces, 
one for the crewmembers and one for the 
instructor. The instructor can "interact with 
the CPT in three operational modes: Pre-Run, 
Run, and Post-Run. The interface has been 
designed to facilitate moving back and forth 
among any of the three modes (See Figure 1). 
The Pre-Run mode occurs prior to the training 
(simulation) session and allows the 
instructor to select and edit a scenario for 
training and complete setup and initial- 
ization procedures. The Run mode is the 
actual training period during which the 
instructor monitors the crewmembers' actions 
and injects changes into the session. The 
Post-Run mode occurs after the training^ 
session in which the instructor can initiate 
a limited review and analysis. All sessions 
mu^st go through some minimum initialization 
via the Pre-Run mode of the user interface. 
Depending on the desired degree of 
interaction, initialization can vary-widely. 
After the appropriate initialization, t^Sw 
instructor will enter the Run mode to begin^ 
the training session. He will be provided 
with commands that allow him to abort the 
session and return to the Pre-Run mode or 
interrupt for a time period and then resume 
within the Run mode itself. Once the run is 
complete, the instructor enters the Post-Run 
mode in which he may conduct limited review 
and analysis or he may return to the Pre-Run 
mode to initiate another training session. 



Separate areas or windows will be set aside 
for various types of- information. 

The largest area at the top of the 
console is reserved for two major functions. 
In the Pre-Run mode this area is used for 
displaying tables of values describing flight 
path characteristics of a selected scenario. 
During the Run mode, this area shall be 
dedicated to the real-time display and update 
of a set of variables denoted as real-time 
parameters. During Post-Run mode, this area 
shall also be used to reflect real-time 
parameter updates, but shall reference only a 
subset of those parameters used during the 
Run mode. 

The Command Menu area, at the bottom 
left, will be used in all three modes to ' 
display the commands, in meny^ form, available 
to the instructor for selection. As the 
instructor transits from one menu to another, 
.this area will display the new menu. 

The area located to the right and top of 
the menu consists of two lines displaying 
faults and pilot-controlled status 
information. This area will b.e active only 
during Run mode and will be empty during both 
Pre-Run and Post-Run modes. The first line 
of the area, the Faults Injected line, will 
indicate any malfunctions or faults currently 
in effect. The second line shall display the 
current status of both the Pilot Steering 
mode (manual/auto) and Terrain Avoidance mode 
(on/off). 



EXECUTIVE MENU 




Figure 1 
Instructor Interface 



COMMANDS AND DISPLAY LAYOUT 

Within any given mode, the instructor has 
a set of commands available for carrying out 
his responsibilities. The command menus and 
other information will be presented on an 
alphanumeric CRT laid out as in Figure 3. 



In all three modes, prompts to the 
instructor for input and notification of 
errors will appear in the Prompts and Error 
Messages area. If the instructor 
inadvertently enters any form of unacceptable 
input, he will be notified, told the nature 
of the mistake, and -requested to reenter the 
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'desired input. The Instructor Input line is 
active in all three modes and is the only 
line able to receive and echo input from the 
instructor. Ihe bottom line of the terminal 
is hardware-controlled and outputs terminal 
status information. 

the tables which can be viewed by the 
instructor during the Pre-Run mode are used 
for purposes of preview, editing, and 
scenario generation. Ihe information 
contained in the tables describes the selected 
flight plan values and cover such items as 
destination points, air speed, missile 
information, etc. these entries may be 
altered by the instructor to develop a 
different flight plan and can even be 



by typing the number of the desired command. 
All keyboards input will be displayed in the 
instructor input area of the screen. Once 
the number has been input, the instructor 
merely has to hit the CARRIAGE RETURN (CR) 
key to activate the selected command. If the 
command then requires additional input such 
as a numeric value before activation can be 
completed, the instructor will be given a 
prompt indicating what information is 
required and what will have to be typed in. 
Editing keys will be provided to allow the 
instructor to recover from typographical 
errors. Once the numeric value desired is 
displayed in the active field, the instructor 
again hits the CARRIAGE RETURN for final 
input into the system. This menu-select- 



PRE-RUN: SCENARIO 

RUN: REAL-TIME PARAMETERS 







1 
1 
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FAULTS INJECTED 
PILOT STEERING AND TA MODE STATUS 


COMMAND MENU 


1 

.J 
f ■ 
1 






PROMPTS AND ERROR MESSAGES 




I 
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INSTRUCTOR INPUT J 



TERMINAL STATUS 



Figure 3 
Instructor Console Layout 



"trimmed" down to create a much shorter 
scenario. 

A wide variety of variables are displayed 
during the Run mode. These variables relate 
such information as aircraft status, 
navigational system accuracy, INS drift, 
crewmember input, system state values, etc. 
Once displayed > the values of these- variables, 
are continuously updated at 5-second 
intervals • 

A menu of available commands will be 
presented in the Command Menu area. The 
command labels will "be numbered on the 
display and may be selected by the instructor 



to -activate design is aimed at minimizing 
the amount of typing required. by the 
instructor. It also' has the advantages of 
always displaying the available commands, 
eliminating any errors due to command-typing 
mistakes, and allowing the user, via the 
numerical select procedure, to quickly access 
and activate commands of interest. 

SYSTEM EXECUTIVE ^ " 

When the mainframe is brought up and 
initialized, the first thing that the 
instructor sees is the EXECUTIVE MEN (See 
Figure 4) which consists of the three simple 
mode selections of Pre-Run, Run, and Post 
Run. In the hierarchy of menu organization, 
the instructor can always return to the 
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EXECUTIVE menu to change modes any time 
during a training run. From this menu, the 
instructor may choose to enter any of the 
three modes or exit the training program 
entirely. In the event the instructor 
activates the Run or Post-Run modes without 
having specified a working scenario, a 
default value- will be assumed and scenario #1 
will be loaded into the system as the current 
working scenario. Once the instructor 
selects any of the modes, the terminal 
displays the master menu for that mode. 
Using the commands in these master menus, the 
instructor can move to other subsets of 
commands that allow hitn to carry out any of 
the tasks required in the selected mode. At 
any time, the instructor can activate the 
RETURN TO EXECUTIVE command available in each _ 
of the roaster menus and return to EXECUTIVE 
menu to select a new mode. 



EXECUTIVE MENU 

1 ) PRE-RUN 

2) RUN 

3) POST-RUN 

4) EXIT 



Figure 4 
The Executive Menu 



OPERATIONAL MODES AND FORMAT OF COMMANDS 

The instructor will be provided with a 
set of seven pre-canned scenarios and 
material describing each of these, including 
navigational charts. The seven scenarios 
will be permanently numbered 1 through 7 and 
will be available at any time for the 
instructor's -use. There is additional space 
for three more scenarios, numbered 8 through 
10, which can be generated by the instructor 
through modification of any of the available 
scenarios and stored in these last three 
slots. Thus, there are a total of ten 
possible scenarios, seven permanent ones 
(numbers 1 through 7) and three replaceable 
ones (numbers 8 through 10). 

* Upon entering the Pre-Run mode, one of 
the first requirements of the instructor is 
to select a scenario by recalling one of the 
ten that are stored. Once recalled, all of 
the initialization values stored under that 
scenario's number will be loaded into the 
mainframe. This scenario now becomes the 
"working" scenario and pan be used for the 
training session as is o? edited to create a 
new, Modified working scenario. Note that if 
a working scenario is not explicitly 



selected, the default working scenario will 
be scenario #1* 

The canned scenarios can be modified by 
the Instructor. Modifications to the working 
scenario are divided Into two categories: 
(1) direct alteration of specific flight leg 
characteristics (such as aircraft speed, 
altitude, wind velocity, etc.), and (2) 
scenario reduction. The first editing 
capability is accomplished by identifying 
specific flight legs of the working scenario, 
and replacing current flight plan values with 
different values. Scenario reduction is 
achieved by specifying a contiguous set of 
flight legs that exist in the working 
scenario. This set becomes the new "reduced 11 
scenario, and retains all flight 
chajracteristics-as defined for the « 
corresponding legs of the original working 
scenario. All editing functions which create 
a new scenario description cause the original 
working file to be replaced by the new 
scenario which then becomes the current 
working scenario. Only the working scenario 
stored in the mainframe is changed by the 
editing functions; the scenario stored on the 
disk remains unchanged. When the instructor 
has completed editing a scenario, he may 
choose to save a copy of the working scenario 
for future retrieval. If the copy is. not 
saved, all modification and initialization 
which has occurred during* Pfce-Run will be 
lost when the CPT program is exited. If no 
editing has occurred, it'irs not necessary to 
save the working scenario since it is merely 
a copy of one of the scenarios already 
stored. 

PRE-RUN COMMANDS 

The four commands which are available for 
selection during Pre-Run mode are listed in 
menu form in Figure 5. When activated, each 
command will either produce a new menu of 
sub-commands, or will cause a request for 
additional information in the form of a 
prompt. In the following sections each 
command will be addressed in detail with the 
corresponding display changes and required 
inputs clearly outlined. 



PRE-RUN MENU 

1) SELECT SCENARIO 

2) EDIT/PREVIEW SCENARIO 

3) SAVE SCENARIO 
—4) RETURN TO EXECUTIVE 



Figure 5 
Pre-Run Menu 



ERLC 



82 



Activation of the SELECT SCENARIO command 
is required before any manipulation of any 
% scenario other than scenario #1 may be 
accomplished. When the SELECT SCENARIO 
command has been activated (by having the 
instructor type 1 CR ), the command menu 
will remain on the screen, but a prompt will 
appear requesting a scenario file number* 
The instructor must then input a number from 
1 to 10 (representing the ten scenario files) 
followed by a CR . This will defjt-ne the 
scenario file which will become the current 
working scenario. Any previous working 
scenario will be replaced b\ this selection. 
Should the instructor declde Nnot to define a 
(new) working scenario, he may^nter a CR by 
itself. This will be accepted as a negation 
to the original command and control will 
resume at the command menu level. Any 
illegal inputs-will be flagged as such in the 
error and prompt area of the console and the 
instructor will be requested for new input. 

By entering 2 CR , the instructor 
activates the EDIT/PREVIEW SCENARIO command 
which causes a new command menu to be 
displayed (see Figure 6). Each command in 
the menu is related to the display of 
particular data tables or values providing 
information describing certain attributes of 
the working scenario. These tables or values 
can be requested for purposes of preview by 
the instructor, or can be edited for 
production of a new scenario. 



EDIT/PPEVIEW SCENARIO 

1) FLIGHT PLAN 

' 2) SRAM TARGET TABLE 

3) DESTINATION TABLE 

4) FIX-POINT TABLE 

5) MISSILE STATUS 

- 6) RETURN TO PRE-RUN 



Figure 6 
Edit/Preview Scenario Menu 



The first command on the menu is the 
FLIGHT PLAN command (see Figure 6). Entering 
a 1 CR will cause the first page (11 lines) 
of the Flight Plan table to appear in the 
scenario display area (upper window), and the 
FLIGHT PLAN menu to appear in the Command 
Menu area. The Flight Plan table lists the 
destination points and flight characteristics 
associated with each flight leg in the 
working scenario. The FLIGHT PLAN menu 
contains a set of commands allowing editing 



and scrolling of the flight plan. , The 
editing commands will explicitly call 4 out 
only those columns in the flight plan which 
may be edited. Whenever the instructor 
decides to change one entry in the table, all 
other entries which are affected by that 
change will be automatically replaced and 
updated on the display. 

The second command available* in the 
EDIT/PREVIEW SCENARIO menu is SRAM TARGET 
TABLE (see Figure 6). Activated by a 2 CR 
keystroke input, the SRAM Target Table is 
displayed in the scenario display area, and 
the SRAM TARGET TABLE menu appears in the 
command menu area. All targets in the 
scenario are listed in the table by target 
number, with the flight leg destination 
number and target characteristics such as 
latitude, longitude, and elevation displayed 
in the corresponding row. The only two 
columns which are available for editing by 
the instructor are SAIR (Safe And In Range) 
ENTRY and SAIR EXIT. 

The Destination Table (see Figure 6) is 
called up by selecting 3 CR from the EDIT/ 
PREVIEW SCENARIO menu and presents the 
working scenario flight plan by listing each 
destination point, type of destination point, 
latitude, longitude, elevation, and planned 
time of arrival. The DESTINATION TABLE menu 
will also be displayed in the command menu 
area. No editing of specific values in the 
Destination Table is possible, but the 
instructor may create a new scenario by 
reducing the current working scenario by 
selecting a contiguous sub-portion of it. 
This sub-portion will then become the new S 
working scenario. * 

The Fix-Point Table (4 CR) is only 
available for presentation; no editing may 
occur. This table will list all fix points 
which may be accessed within the limits of 
the working scenario. The points will be 
listed sequentially by number with the 
corresponding latitude, longitude, and 
. elevation of each point displayed as well. 
This table wiJrt. usually exceed display 
capacity in silze and will require a paging 
capability. 

By entering a 5 CR , the instructor can 
change Missile Status. This parameter is set 
to either 1 (Fully Aligned) or 2 (Power Off) 
in the scenario. The Fully Aligned status 
will cause the elimination of the required 
warm-up time of missiles and weapons within a 
scenario. It will initialize the scenario 
with missiles fully targeted (powered, armed, 
and in "GO" status) and Mills fully powered. 
The Power Off status assumes no warm-up of 
missiles or weapons has occurred. Upon 
entering the command, no change In the menu 
occurs and nothing is displayed in the 
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scenario area. Rather, a message appears in 
the prompt and error area indicating the . 
current value' of the missile status parameter 
and requesting a new value. Entering 
anything other than 1 CR or 2 CR will 
cause art error message to appear and a 
request for new input. Entering CR only 
will negate the command giving control to the 
EDIT/PREVIEW SCENARIO menu. 

When the instructor selects the RETURN TO 
PRE-RUN (6 CR), the EDIT/ PREVIEW SCENARIO 
command menu will clear from the display area 
and control will return to the PRE-RUN master 
command menu (see Figure 6). 

If the instructor wishes to save the 
current working scenario and its 
corresponding parameters^or later 
retrieval/u^age, he raust-^fctivate the SAVE 
SCENARIO command by entering 3 CR in the 
PRE-RUN master menu (See Figure 5). A prompt 
will appear requesting a scenario file number 
for 8 to 10 (representing the three 
replaceable files available for storage). 
Once selected, the old contents, of the file 
will be, destroyed and replaced by the current 
working scenario. If the instructor decides 



not to save the working scenario, he may 0 
enter a CR by itself. This will negate the 
command activation, and control will return 
to the PRE-RUN master command menu. All 
illegal inputs will be flagged as such to the 
instructor and he will be requested^ for new 
input. 

Activation 6f this command will clear the 
PRE-RUN master command menu from the display 
and control will return to the EXECUTIVE 
command menu (See Figure 5). 

RUN MODE 

The instructor activates the' Run mode by 
entering 2 CR from the EXECUTIVE menu. 
Upon entering this command, the training * « 
session (simulation) starts;. 

, Throughout the training' session, a set of 
real-time parameters reflecting aircraft, 
navigational, and command information are 
displayed in the u^per wincfow of the console 
and updated every 5 sec -(see Figure 7). The 
simulated time is displayed at the top, 
right-side. Below that is the aircraft data 



AIRCRAFT DAT7I 

TATi — N47-00.00 

LONG: W119-00.Q0 

Navigational data; 

LAT1 N 47-15.10 
LONG: Wl 20-1 0.00 



[TIME 14:12:30? 



LAST COMM/USPlSSUEq 
RFP I K3 : XHAIR 10,12 
N-IKB2 MDFY 24 



TH: 305 ALT: 21500. DEST: 16 B28-TGT 
TK: 300 W/V: 300/50 - - TAS: 420 



TH: 300 ALT: 22100 PTA: L 00:16:13 

TK: $10 4 W/V: 310/30 



RNMP: .RANGE/SCALE 75 
PANEL : WCP — LP 1 ,2,3 WPN PWR 



RUN MENU 

1) FAULT INJECTION 

2) AIRCRAFT MANEUVER 

3) TAL RECOGNIZED 

4) TERRAIN AVOIDANCE MODE 

5) FREEZE 

6) RESUME 

7) RETURN TO EXECUTIVE 



RNMP-FAIL WCP-FAIL 

PILOT STEERING: MANUAL TA MODE: OFF 



(PROMPTS AND ERROR MESSAGES) 



(INSTRUCTOR INPUT) 



(TERMINAL STATUS) 



Figure 7 
Run Display 
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cons^stjiig ofT) latitude and longitude, true 
heading and ground track, altitude and . 
winu/*eloci*y; current destination point, and 
true air speed. Next is, th e navigational- 
data reflecting values/tor the prime' 
navigational system, (These parameters are- 
latitude arid longitude, true heading and 
^ ground track, altityfie and wind/velocity, and 
the planned time / arrival 'error (early or 
late). Navigatj/nal errors can be. determined 
from these pa^meters by subtracting 
corresponding values of the aircrlft and 
navigational] data. 

The bottom portion of the window displays 
the last command issued by the crewmembers — 
separately for the three major panels: 
RN-IKB, N-IKB, and JUMP. The -.last command 
Issued for the remaining set of panels is 
designated by. a panel label followed by the 
command given. 

v This display of real-time parameters 
allows the instructor to have immediate, 
completely updated information as to the 
aircraft state and crewmember actions. 

In addition to the real-time parameter 
display, entering the Run node causes the RUN 
menu to be displayed in the command menu 
area. There are seven commands associated 
with the Run menu and the following sections 
will explain each of these in detail. - 

- RUN MODE COMMA NDS 

The instructor has the capacity to inject 
malfunctions-, or fauljts, at any time during 
Run mode (see Figure 7). This allows him to 
cause failures to occur at critical times 
corresponding with crewmember activity. The 
fault specified will occur at the moment of 
select idn. He also has the ability to 
terming a given malfunction that is 
^currently in progress. " \- 

Upon entering a 1 CR , the FAULT 
INJECTION menu will appear in the Command " 
menu area (see Figure 8). 'The menl provides 
four fault commands and la RETURN T(5 RUN 
command . 



FAULT INJECTION 

1 ) RN MANAGEMENT PANEL 

2) WEAPONS CONTROL PANEL 

3) DOPPLER RADAR 

4) OAS BUS FAILURE 
- 5) RETURN TO RUN 



Figure 8 
Fault Injection Menu 



Selection of the RN -Management Tanel 
command. (1 CR ) will cause the current 
status of the RN Management Panel (RNMP) to 
be indicated in the^ prompt and error area 
.[ 1 (FUNCTIONING) , .2 (FAILED)] along with a 
request for input. If the current status is 
FUNCTIONING and the instructor elit'ers a 2 
CR, the RfWP will immediately cease to < 
function, and the message RNMP-FAIL will 
appear on the Faults Injected line of the 
* instructor's console. If the current status 
is FAILED and a 1 CR is entered, the RWP 
will begin to function again and the 
RNMP-FAIL message will be erased-. If a CR 
is^input by itself, the command will negate 
and the v fault status will remain unchanged. 
An input o£ any value other than 1 or 2 will 
cause an error message 'and a request for new 
input. 

Validation checking will occur by the 
system pripr to activating any fault. Since 
no other fault can occur simultaneously with 
an OAS bus failure, the system will check the 
OAS bus^ status before initiating other 
failures. If the OAS bus status is FAILED 
and the instructor attempts to inject an RNMP 
failure, an error message will appear in the 
prompt and error area and rnmp status will 
remain FUNCTIONING. 

Weapon Control Panel failure (see Figure 
^8) is activated by entering 2 CR U . it 
operates exactly as the RN MANAGEMENT PANEL 
command except that tfie message WCP-FAIL 
appears in the Faults Injected area of the 
console . 

The Doppler Radar failure (se^Figuke 8) 
^ is activated by entering 3 CrC^> i t ' 

operates exactly as the RN MANAGEMENT PANEL 
- command except that the message DOPPLER-FAIL 

appears, in the Faults Injected area of the 

console. 

The OAS Bus Failure (see Figure 8) 
differs from the other fault commands in that 
once this fault is irijected, it. must remain 
FAILED for the rest of the training session. 
Thus, there is no need to' prompt for an input, 
value for the command. 

When 4. CR is entered, the system will ' 
check the status of the other three panels. 
If any of them are currently FAILED, an error 
message .will appear and, the command will be 
ignored, if all three of the other panels 
are currently FUNCTIONING, the/ system -flien • 
checks the OAS Bus status. If it is 
currently FAILED, another error message will 
appear and the command will be ignored. 
Finally, if all three panels and the OAS Bus 
status are currently FUNCTIONING when the 
command is activated, the system changes the 
OAS bus status to FAILED and the message OAS 
BUS-FAIL appears in the F&ults Injected area 
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of the console. " Once the^ instructor has 
injected this failure, he will have* no 
further* need to return to the Fault Injection 
menu throughput tjie remainder of the training 
session. * * 

Entering 5 CR (see Figure 8) will clear 
the FAULT INJECTION menu from the display 
area and control will return to the RUN menu 
(see Figure 7). 
/ 

During the progression of the training 
session, the instructor has the capability of 
acting as a simulated pilot in taking the 
aircraft through changes in speed, altitude, 
heading", and steering "mode.. He can also 
change the wind/velocity (direction and 
magnitude) and the alternate navigational 
Heading error. j 

Upon entering T^CR , the AIRCRAFT 
MANEUVER menu will be displayed in the 
' command menu area (see Figure 9). This menu 
consists of seven commands for maneuvering 
the aircraft, changing conditions, or 
returning to the RUN menu. 



input, the aircraf t^will J?egirt a constant 
2000 f t/min climb or descent until it reaches 
the desired altitude, fke real-time ALT 
parameter will reflect the upda.teja a£tit.ude.„' 



AIRCRAFT MANEUVER 

1) TAS 

2) ALTITUDE 
— HEADING 

^J) PILOT STEERING MODE 

5) W/V 

6) ALTER NAV HEADING ERROR 

7) RETURN TO RUN 



X Figure 9 
Aircraft Maneuver Menu 



When TAS is selected by the instructor 
with a (1 cffTT~a prompt will be issued for 
input of a new air speed value. The current 
air speed value is displayed continually and 
updated in the real-time parameter display 
window. 

Once a new value has been successfully 
input, the aircraft will begin a constant 
acceleration/deceleration until it reaches 
the desired velocity. The real-time 
parameter will reflect this .change. 

As with the aircraft velocity, when the 
! ALTITUDE command (see Figure 9) is activated 
by entering 2 CR , a prompt will be issued 
for input of a new altitude. The current 
altitude is another real-time parameter that 
is displayed in the upper window. 

/ 

Once a new altitude has been successfully 



. After a new altitude Has b6eV input by 
the" instructor, the aircraft remains at tha^ 
new altitude mntil 
instructor. 



further input from the 



The instructor has the capability to 
input a new heading by selecting a 3 CR 
command (see figure 9). Before changing the 
heading, the system will check* the Pilot 
Steering Mode. In order for the pilot 
(instructor) to manually change the heading, 
the Pilot Steering Mode must be MANtJAL. If 
the Pilot Steering Mode is AUTO, no change in 
heading can be initiated by the instructor, 
an error message will appear, and the command 
will be ignored. If the Pilot Steering Mode 
is MANUAL, the HEADING Command will be \ 
initiated. ' ' 

After successfully entering a new 
heading,. the aircraft will begin a turn of 
constant radius until the new heading is 
reached. Once attained, if this heading does 
not take the aircraft to the expected 
destination point, it will remain in effect 
until a new heading value is issued. 

The Pilot Steering Mode can be changed 
(see Figure 9) byentering 4 CR - . The 
current value of the mode ^l(AUTO), 2 
(MANUAL)] will be displayed in the prompt and 
error area along 'with a prompt to enter a new 
value. Entering a value pther than 1 or 2 
will cause an error message to appear and a 
prompt for a new inputs The current setting 
of the Pilot Steering. Mode is also displayed 
on the console below the Faults' Injected 
line. At the start of the training session, 
the Pilot Steering Mode will default to AUTO. 

Entering 5 CR allows the instructor to 
update the wj.nd/ velocity (direction and 
magnitude). A prompt for a new value of 
wind/velocity will appear* The new value of 
W/V entered by the instructor will remain in 
effect until the next turn^ipoint in the 
scenario is reached. After the turn point 
the next wind will be obtained from the 
(scenario) flight plan. AH changes to 1 
wind/velocity will be reflected in the 
real-time W/V parameter value. v* 

The instructor can change Che Alternate 
Navigation Heading Error (see Figure 9) by 
entering 6 ' CR . A prompt for a new value, 
"not to exceed +5, will appear. Any illegal 
input by the instructor will be flagged with 
an error message and a prompt for a new inputv 
will appear. 

If the Alternate Navigation system is the 
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pripe navigational model , the instructor can 
determine Cne current alternate navigation 
heading error by subtracting the" real-time TH 
values for aircraft ,and navigational data. 
Upon entering* a new acceptable value for the 
alternate navigation heading ejrror, the 
alternate navigation system will replace the 
old error with- this new value and update* the 
TH parameter to reflect the new error. Any 
other navigation parameters affected by the 
change (such as TK) will also be updated. 

If the alternate navigation system is not 
the prime navigational model, the instructor 
cannot determine the current alternate 
navigation heading error. He can still enter 
a value within the +5 limit and t lis new 
error will be stored in the alternate 
navigation system replacing the ol^l value. 
However, the aircraft and the current prime 
navigational model 'will not act upon this new 
value until the alternate navigation system 
is selected as the prime navigational model. 

Entering 7 Cg will replace the AIRCRAFT 
MANEUVER menu with the. RUN menu (see Figure 
7) and pass control on to it. 

When missiles and weapons are powering up 
and reach the Transfer Alignment (TAL) mode 
(see Figure 7), a TAL REQD message will 
appear on the crewmember's display. In order 
to complete transfer alignment, the > 
instructor must issue a TAL RECOGNIZED 
command. This is done by entering 3 CR 
from the RUN menu. Once entered, TAL is 
completed and the missiles enter Fine 
Alignment (FA) mode. * If the TAL RECOGNIZED 
command is entered before either the TAL mode 
is reached or the TAL REQD message appears, 
it will be ignored and have no affect on the 
system. _ , 

The Terrain Avoidance (TA) mode affects 5 
the width of the sector that appears in the 
radar video display. There are two values 
for the mode: 1 (OFF) and 2 (ON). Upon 
entering 4 CR , (see Figure 7) the current 
value of TA will be displayed along with a 
prompt for a new value. Entering a value 
other than 1 or 2 will result in an error 
message and prompt for new input. 

The current TA mode is also displayed on 
the console below the Fault Injection line 
following the Pilot SteeTing Mode display. 
Any change in the TA mode will be reflected 
on this display. At the start of the 
training session, the JA mode will default to 
OFF. * 

The FREEZE command (see Figure 7) allows 
the instructor to halt the training session 
at any point without destroying the validity 
and consistency of the simulation. The 
complete session can be frozen at an instant 



in time, allowing the instructor to go to the 
crewmembers and point out a sequence of 
operations or recent commands that were in 
error or inadequate. 

The instructor "freezes" the session by 
entering 5 CR . At that instant, all 
training and simulation procedures for 
crewmember activity will halt. ; The 4 
instructor may interact verbally with the 
crewmembers and advise them of any problems 
they may be experiencing. 

Once the instructor has finished* his 
verbal instruction during the ''freeze 11 , he 
can then resume the simulation at precisely 
the point at which it was stopped, and phe 
training session can' continue as planned from 
that point. 

To continue a "frozen" session', the 
instructor enters 6 -CR (see Figure 7). The 
training session resumes from the exact point 
at which it was frozen. 

POST RUN MODE 

The major purpose of the Post-Run Mode 
(see Figure 4) is to provide a limited review 
of the 'currently defined worjcing scenario. 
The instructor may specify an arbitrary point 
in the scenario as a. start time for review. 
Once selected, he may begin a review of the 
scenario by activating the RESUME command. 

When the review is in progress, the 
.instructor may stop the review at any time. 
This will enable him to make any notes of the 
scenario without missing critical* material or 
allow him tQ^define a new starting point. ? . 
Thus, when used in conjunction with defining 
start. ^ijitB^jajoi-j^ai^ng the .review* process, 
a particular sequence can be replayed many 
times. 

» 4 

POST RUN COMMANDS 

Four commands A are available 'during the * 
Tost-Run mode and ar« presented in menu form * 
in Figure 10. Only the DEFINE START POINT 
command requests additional input .from the 
instructor, and none of the commands invoke a, 
second level of command menu. > * 



P0ST- 


RUN MENU 1 




1) 


DEFINE START POINT 




2) 


FREEZE 




3) 


RESUME 




4) 


RETURN TO EXECUTIVE 








* 
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Figure 10 






Post-Run Master Menu 
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Activated, by a 1 dk , the system will 
display the current .start ^time (which 
defaults to the beginning 6} the flight 
plan). The instructor wilf-^then be pronpted 
fof a new start time and must enter a value 
that is between the start and end of the 
flight scenario. If he enters an illegal 
•value, an error will be flagged, and he will 
be requested for new input. Once a new start 
time is successfully entered, it will become 
the point at which the review process will 
start when the RESUME command is activated. 

v The' FREEZE command ,is only meaningful 
during the review psocess* Activated,, by a 
keystroke sequence of 2 CR , this command 
t/ill stop the rev^ew^ng process. v 

When RESUME command is activate^*, the 
reviewing process will be started or 
restarted. t The scenario wi^l begin at the 
currently defined -start po*nt, or freeze 
point if "the session was frozen, and will 



continue along -as though the aircraft was 
following the flight path perfectly. Radar 
video will correspond to the exact flight 
path with MFD formats available for 
inspection. No on-line interaction of 
crewmember commands will be available except 
for radar video presentation format. Once 
the post-run. review is completed, the 
instructor can return to the exective menu b> 
inputing a 4 CR command. ^Activation of this 
command will clear the POSt-RUN master 
command menu from the' display and control 
will return to the EXECUTIVE command menu. 

HARDWARE 

The PTT has been designed in modules to 
facilitate modifications as the OAS itself w 
changes and as additional OAS trainer 
capabilities are identified. The five 
modules * — independent subsystems — of the 
hardware configuration are depicted in Figur 
4 and described below. They illustrate the 
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use of off the shelf equipment. 

The Mainframe "Processing Subsystem (MPS) 
consists of a minicomputer with -disk drive, 
aagnetic tape derive y and system console (see 
Figure 11). % The disk is used for real-time 
storage and ^retrieval of radar imagery data; 
the magnetic tape is used for transfer of 
'software and updates of terrain data. 
^' V: - * 

* The MPS is the central' processor for the 
PTT. ^t is responsible for logical and 
numerical processing plus cdntrol of all 
other subsystems. The PTT Software^ runs on 
the MPS.' 

The virtual console is a CRT required for. 
the PRIMOS Operating System. The disk drive 
is required for the operating system, for 
software and data file storage, and for 
real-time access to display files for 
synthetic radar imagery updates. ' The 
magnetic tape drive is required for loading 
terrain files and software updates on the 
operational systems in the field. *A11 
peripherals are standard PRIME equipment. 

All' peripheral devices on the PRIME 55.0 
CPU (disk drive, magnetic tape drive,- and 
virtual console) have standard interfaces 
supported by PRIME hardware and software. 
» ** 

The mainframe processing subsystem 
capabilities are listed below for the PRIME 
550' CPU. " 

PRIME 550 CPU * 

32 Bit CPU Architecture 

128 Registers. - • 

512 K Byte Error Correcting Code Main 

Memory (expandable to 2 M Byte) 

1 K Word Gache 

Single/Double Precision Floating Pofnt 

INSTRUCTOR'S CONSOLE-" SUBSYSTEM (ICS). 

The Instructor's, Console Subsystem (ICS) 
serves as a system monitor and control 
interface for the instructor. It provides 
for interchange 0 f textual data with the Part 
Task 'Trainer control programs executing in 
the MPS., The following data are displayed at 
the Instructor's Console on an alphanumeric 
CRT display: 

Flight Parameters 3 
Mission ✓Data • . 

t Status of OAS-Emulated Subsystems 
Operational Faults 

-Alphanumeric data and Control codes are 
in;put through a keyboard. 

Functions which can be exercised are: 



Trainer Session Control 
Alteration of* Flight Parameters 
Modif ication.of *0AS-Emulated Subsystems 
Status 

Fault Seeding 0 ■ 

Recovery'from Operational Faults 

The ICS is a SOROC IQ 140 CRT terminal (See 
Figure 12). 

* * *- 
GROWTH POTENTIAL 

We realized that if we did not build a 
growth capability into the basic PTT design, 
• we would start at square one' again if furture 
considerations required an expansion of PTT 
capabilities. Therefore, growth potential 
was a high priority jronsideration throughout 
the entire program/^ This requirement for 
growth* potential spilled over into the design 
of the instructor station as, -well. . 
* <* * 

First, the instructor station EXECUTIVE ' 
software was designed "with this growth 
capability in mind. For example, we realized 
that we "would eventually have to include more 
malfunctions in the PTT. Therefore, t.he 
instructor software had to be able to handle 
these additional malfunction requirements as 
well. Therefore, the EXECUTIVE so'ftware ' 
design included a group of unused 
; , input/output flags. Thus, when an addition 
is made to the PTT software, the EXECUTIVE 
routines will not have to change'.. This will 
greatly reduce the time and cost of software 
development. 

* ' ' ~ * * 

The system hardware also lends itself to 
expansion^ A 30Z growth capability was 
designed into the PTT mainframe computer. If 
expansion requires more than the planned pad, 
we can add a 50Z memory capability by 
purchasing a $12_,000 memory board. 

♦ The SQR'OC IQ 140 terminal (see Figure 12 \ 
also provides us with tremendous growth 
potential. We have not even begun to take 
advantage of the special functions available 
on the SOROC. ' , , * . 

Finally, the instructor's command formats 
and displays are* ideally suited to* expansion. 
The single keystroke command format can . 
easily handle doubledigit or even triple- 
digit commands. In addition, the 
instructor's displays can be easily "changed 
to meet growth requirements. For -example, if , 
we were to add mote faults than could fit on 
the Fault Injection Menu (Figure 8), we could 
simply scroll this menu just as' the , - 

destination tables are already, scrolled, in 
/ the present configuration. " v 
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The B-52 Offensive Avionics System ,( OAS) 
Part Task Trainer is a trainer that is 

designed to focus on procedures training; 
specifically those procedures unique in 
operation of he n*w H-52 Offensive Avionics 
System. With emphasis on procedures 
training, the instructor plays a major role 
in the training of the students. The OAS PTT 
'was a two-.user interface, one for the 
crewmembers and one tor the instructor. The 
instructor's interface is designed to free 
the insfructor to interact with the student* 
at all times. Specification requirements of 
the instructor stat'on included; ease of 
operation, rapid scenario setup, access to 
students, easy student and system monitoring, 
the growth capability. With these - 
requirements in mind, an instructor station 
and interface was designed to rake the 
instructor's job simple and effective. 




TRAINING EFFECTIVENESS EVALUATIONS OF * 
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Substantial increases in simulator 
technology have dramatically increased the 
scope and potential of simulator training. 
- Ultimate simulator traiifing- -effectiveness, 
hcwever, is not only a function of a 
(levies' s capability to simulate training 
tasks accurately, but also of its ability to 
operate as an effective instructional tool. 
Its effectiveness as an "instructional tool 
depends upon Instructor /Operator Station 
(IOS) factors such as instructor /operator 
workload, performance monitoring .and evalua- 
tion capabilities, and training task/mission 
■set-up. Effective design and proper utili- 
zation of the IOS can affect the potential 
and achieved effectiveness of simulator 
training. 

Advances in simulator technology, and 
attempts to f incorporat^f state-of-the-art 
instructional 'technology, have resulted in 
increasingly corplex and sophisticated 
instructor /operator stations. Onfortu- 
nately, this application of new technology 
has not always led to advanced in instruc- 
tional effectiveness of efficiency. This ; 
shortcaning, and the potential importance of 
effective IOS functioning, highlight the 
importance of evaluating IOS design and 
innovations in simulator 4 instructional 
support systems. However, accurately 
assessing the effectiveness of simulator 
instructional support systems poses a set of 
problems that , often are nc5t solved by 
traditional approaches to evaluating simu- 
lator training effectiveness. 

This paper discusses seme of the 
problems encountered in attempting to 
evaluate the effectiveness of simulator 
instructional support systems and provides"" 
some potential solutions to these problems. 
Three areas are addressed: selection of a 
suitable evaluation model and sensitivity of 
performance measures; 'evaluation cbjectives; ( 
and instructor training. 

Evaluation Models , ' 

Additions or* modifications to training 
* systems usually nust be justified by showing 
that they either increase training effective- 
ness and/or that they somehow reduce 
training costs while maintaining some c(iven 
level of effectiveness. This is especially 
true *if the addition or modification repre- 
sents a prototype x or proof^of -concept 
training device or instructional aid. 

In general, evaluation models used to 
assess training effectiveness can be£#rouped 



iitto one of two categories: analytical 
. models and experimental. models. Analytical 
models span a broad variety of procedures 
ranging from simple questionnaires and 
opinion surveys to well-developed, highly 
- structured rating scale evaluations. 

Analytical models usually do not involve 
direct measures of studept or instructor 
performance. Instead, they rely upon esti- 
mates, ^judgements, or opinions that are 
based on direct observation or experience . 
Experimental models rely on direct measure- 
ment of performance. They normally involve 
comparing the performance of one group of 
students trained with the addition/modifi- 
cation to the performance of another group 
of students who are trained without the 
addition or modification . * Under this model, 
differences in performance can be attributed 
to the influence of the instructional addi- 
tion or modification provided that other 
factors that influence performance are held 
^ constant for both groups of students. 

A transfer of training evaluation is 
representative of the experimental model in 
which the~ amount of training required to 
attain proficiency with the actual equipment 
is related to the amount of previous 
(simulator) training. Transfer of training 
studies allow simulator training effective- 
ness to be expressed as the amount of actual 
equipment training that can be saved by 
simulator training. Such studies also allow 
calculation of potential cost savings- that 
can be achieved through simulator training. 

Experimental comparisons , Such as 
transfer of training studies , have become 
the^ standard fgr assessing simulator 
training effectiveness. There has been a 
natural tendency, therefore, to apply these 
same experimental models to assess the 
effectiveness of simulator .instructional 
support systems such as instructor/operator 
stations, special instructional features, 
and simulator instructional capabilities. 
From a methodological standpoint , experi- 
mental, model s repre sent the pre f erred 
. approach to assessing training effective-' 
ness. However, 'there are at least two 
factors thdt limit the adequacy* of this 
approach for evaluating simulator instruc- 
tional support systems. 

The first factor concerns the expected 
- ooitributicn of the instructional system 
addition/modification to training compared 
to the ccntributionr made by the overall 



training system. If the relative increment 
in training effectiveness is small, then it 
nay not be detected on the overall system 
level, i.e. , in terms of student perform- 
ance. For exanple, student performance 
typically is affected by multiple conponents 
of the training system such as academic, 
classroom, simulator, and actual equipment 
training. In most cases, the instructional 
support system of a simulator, carpared to 
the overall training system, represents a 
modest influence on student performance. 

• Such small effects require very sensitive 
performance measures and a level of experi- 
mental control and rigor that usually is not 
available for evaluations conducted in 

. operational settings. 

The second factor concerns using 
instructor-assigned grades as a measure of 
student performance . Although instructor 
grades frequently are used in training 
effectiveness evaluations, they normally are 
not very sensitive to actual differences in 
* student performance. Instructor grades are 
used more to manage student progress through 
the training program than they are to 
measure performance. Instructor grades also ' 
tenc^ to be norm-referenced, i.e.,'£hey are 
assigned relative to the average performance 
of other students of comparable training and 
experience. Although grades serve their 
intended {Purpose, they alone usually are not 
sufficient to detect differences in perform- 
ance, especially if those differences are 
small. ' ^ 

The solution of the first problem ' 
requires that the evaluation be conducted at 
a level commensurate with the expected 
increment in training effectiveness. If the 
change or addition to training represents a 
major irrpaot on training, then it may be' 
sufficient- to assess that change at a molar 
level, such as by measuring difierences in 
student performance. On the other hand, if 
the change or modifications represents a 
small iirpact then it will be necessary to 
assess that change at a more -molecular 
level. This may require a change in the way 
training effectiveness evaluations are 
conducted. Experimental comparisons -of 
student performance may need to be replaced 
or supplemented with more analytical compar- 
isons. For exanple, changes in the instruc- 
tional- features of a simulator, which repre- 
sent a small change in the overall training 
system, could be evaluated by establishing a 
list of behavioral objectives that the new 
instructional capability should achieve. 
The evaluation would then assess the extent 
to which those objectives were met* Such 
objectives should be a natural part of the 
design process it§elf and they should 
reflect the original purpose for developing 
or modifying a particular instructional 
feature or capability. Establishing a list m 
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of behavioral objectives would allow a clear 
and. specific statements to be made about the 
expected results of a particular simulator 
feature and also would provide a clear basis 
for assessing the extent to which those 
results were achieved. This information 
could be used to evaluate the, merits of a 
given simulator feature, regardless of hew 
small a oenponent that feature was, relative 
to the overall training system. An assess- 
ment of behavioral objectives also could be 
viewed within the context of a broader scale 
evaluation of student performance. 

The second problem, i.e., sensitivity 
of instructor grades c^n be addressed in a 
number of ways. First, 'a set o£ -objective 
measures of student per f ormance ^opuld be 
developed. However, the time 2nd cost 
requirements to develop good performance 
measures often exceeds the resources avail- 
able for training effectiveness evaluations. 
Second, instructors could be asked to record 
actual deviations* in' performance for a 
selected set of parameters judged to be 
especially critical to successful perform- 
ance of the task being graded. This would 
provide some additional performance infor- 
mation for the purpose of the evaluation. 
It also would allow subsequent instructor 
ratings to be based, in part, on selected 
aspects of student performance. Third, the 
standard four-point grading scale u^ed by 
instructor's could be expanded to a seven- 
point scale by inserting intermediate points 
between each original scale point. This 
would increase the potential sensitivity of 
the middle part of the scale which' is most 
frequently used, but it would not signifi- 
cantly alter the basic format of the scale. 
Therefore, instructors still would have a 
scale they are familiar with, but one thctt 
has greater sensitivity. Finally, automated 
performance - capabilities of simulators 
should be utilized if they are available. s 

Evaluation Objectives 

Related to the topic of evaluation 
models is the issue of evaluation objec- 
tives. Although the overall goal of an 
evaluation may be to assess the training 
effectiveness^ or efficiency of a simulator 
instructional support system, more specific 
evaluation objectives should be formulated. 
Specific evaluation objectives are especi- 
ally important for evaluating instructional 
support systems because the specific proce- 
dures for evaluating such systems, including 
dependent variables, are not as well defined 
as zney axe for evaluations of total 
simulator systems. Specific evaluation 
objectives can provide a conceptual 
framework for the development of specific 
evaluation procedures. Clearly stated, 
specific evaluation objectives als# can be 
helpful in selecting an evaluation model and 
for guiding the development of measures of 



instructional effectiveness. > 

3he development of evaluation objec- 
tives can take 'many forms, however, seme 
criteria that objectives should satisfy are 
as follows* First, objectives should be 
based on the overall* purpose of the 
evaluation* Objectives, therefore, should 
be stated so that their fulfillment will 
•4 provide answers to the questions that 
• originally generated the evaluation* 
Second, objectives, should be operationally 
meaningful in the sense that their fulfill- 
ment will provide useful information* 
Third, objectives should be stated clearly* 
Finally, objectives should be testable 
within the operational context in which the 
evaluation will be conducted. 

Instructor Training 

Another iiqportant factor in conducting 
an accurate evaluation of any instructional 
support system involves instructor/operator 
training. A device must be used correctly, 
and with sufficient knowledge of its limita- 
tions and capabilities, to achieve an 
accurate assessment of its true effective- 
ness. Instructor training also plays a key 
role in user acceptance of new training 
systems. Unfortunately, many evaluations 
begin without adequate instructor/operator 
training* 

The solution to .the problem of 
instructor training is straightforward. 
Training effectiveness evaluations of simu- 
lators and their instructional support 
systems should not be started until the 
instructors/ operators who are going to use 
them are adequately trained. The nature of 
such training should be threefold. First, 
instructor/operators need to know how to 
operate the simulator. Second, they need 
to-be trained in hew to effectively utilize 
the instructional capabilities of the simu- 
lator. Third, .they peed to be trained with 
the procedures that - are going to be used 
during the evaluation. 

' Theoretically , instructor training 
should be a simple task to conplete . 
Hcwfever, it often poses a number of 
problems, including the time required to 
train all of the instructors, scheduling 
.instructor training around the existing 
training schedule, «and rotation of instruc- 
tors in and out of the training conmand. 
Many of these problems could be solved by 
selecting a set of instructors who are not 
expected to transfer away from the training 
squadron 'until* after the evaluation is 
completed. Having a smaller group of 
instructors participate in the evaluation 
would simplify x the instructor training task' * 
and would perhaps al lew enough time to 
provide, them ' with some individualized 
on-the-job tutorials. Alternatively, using 
a suftset. of available instructors will 



reduce the nwfcer of students that can be 
followed at any given time and, hence, will 
increase the duration of the evaluation, it 
also may cause some problems for scheduling 
simulator training events* 

Sunnary 

Conducting training effectiveness eval- 
uations of simulator instructional support 
systems involves all of the problems gener- 
ally associated with assessing simulator 
effectiveness and poses "seme special 
problems as well* This paper addressed some 
of these problems and discussed some poten- 
tial solutions. , 

Listed below are seme guidelines that 
may help in designing and conducting 
training effectiveness evaluations of 
instructional support systems* These guide- 
lines incorporate seme of the problem 
solutions discussed above as well as some 
recomendations not discussed* 

- Ensure that the system to be evaluated is 
sufficiently developed and debugged before 
the evaluation begins. 

- Develop speci f ic evaluation obj ectives 
that can be* addressed with the resources 
available to conduct the evaluation. 

- Select or develop an evaluation model that 
will allow evaluation objectives to be 
satisfied. 

- Ensure that- the evaluation model is 
appropriate for the size of the training 
effect expected to be detected by the 
evaluation. 

- Select or develop a set of measures that 
together will allow all evaluation objec- 
tives to be addressed. 

- Ensure that measures of performance are 
sensitive enough to detect differences 
attributable to using the instructional 
system being evaluated. 

- Develop a conceptual plan for relating 
measures of performance to evaluation 
objectiye^ (this is especially important 
for evaluations that use analytical models) . 

- Ensure \that instryctots/operators are 
adequately tbain^^to^erate the simulator, 
and to ytilizethe capability of the in- 
structional system bfeing evaluated. 
Instructors/operators also should be trained 
with the evaluation procedures to be 
followed. 

- ^Monitor daily the conduct of the. evalua- 
tion and data collection procedures. 
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Following these guidelines will not 
guarantee a successful evaluation and they 
do -not address all of the issues associated 
with conducting a training effectiveness 
evaluation. Hcwever, they may provide seme 
assistance in designing and planning an 
evaluation and they may help minimize seme 
of the problems encountered by previous 
evaluations of instructional support systems. 
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ABSTRACT 

The increasing technical complexity of modern 
systems require the application of new techniques in 
the design of the man-rftachine interface. Stress 
caused by information overload and "underload" will 
not be relieved by incpfeased training or better per- 
sonnel selection. Rethinking' the man-machine dialogue 
is required. Advances liymodeling methodologies offer 
promising possibilities for the future; 3-D Computer- 
Aided Design (CAD) systems offer solutions that are 
available today. Application of one such system, the 
Enhanced Interactive Design System (EIDS), is dis- 
cussed with reference to its use in actual weapons 
system design. 



INTRODUCTION 

Ever more complex technological demands 
for ever more accurate weapons systems have 
made these systems so complex that the oper- 
ating personnel are being pushed toward their 
operational performance limits. Neither pro- 
longed training nor increased personnel selec- 
tivity can ensure the right soluticfas to these 
pressing problems. The use of *huroan engi-* 
neering principles to define the man-machine 
dialogue and the man-machine interface has long 
been stressed as the solution to the need for 
shorter training times and lower qualification 
profiles, with better operating results. How- 
ever, appropriate human engineering principlesr 
have yet to be adequately applied. 

Human factors engineering considers a 
large area that includes not only anthropomet- 
rics, mechanical, and chemical (chemo-physi- 
cal) stress factors but also the whole range of 
man-machine Interactions. These factors im- 
pact the man-machine dialogue through indica- 
tors, controls, and visual and auditory infor- 
mation from the environment and also include 
other sensory data and information that are not 
directly related to the operator's task, such 
as movement, especially accelerations and vi- 
brations, sound, noise, temperature, and the 
whole microclimate. Another factor in the man- 
machine dialogue is the proprioceptive infor- 
mation, I.e., the Information regarding the lo- 
calization of the body and Its extremities in 
space, Its movement, etc. , 

In today's human engineering community, 
our efforts must no longer be primarily ori- 



ented toward further increases in the rate of 
data output. The way toward higher efficiency 
is increasingly dependent on how yelt we suc- 
ceed in reducing stress. Han has become the 
weakest link in the chain. This is mostly be- 
cause engineers and system designers have al- 
ways tried to solve the technical problems 
first. Because of their mechanistic blinders, 
they have hardly ever confronted man-related 
problems unless forced to. Those human factors 
that were recognized and respected were the 
most primitive and easiest to capture, e.g., 
anthropometrics (at least I have no other ex- 
planation why, for a lot of people, ergonomics 
is still understood as a synonym for anthropo- 
metrics). Han is much more complex than simply 
a set of measurements* A lot of physiological 
and psychological knowledge , has been cpllected 
during the past 50 years, but little of this 
has been directly applied to workstation de- 
sign. Actually, having approached the limits 
of man's capabilities, we largely lack the 
methods for applying this knowledge to the en- 
gineering of a modern workstation. 



STRESS AHD INFORMATION LOAD 

One of the major tasks for human factors 
specialists 1s stress reduction. In order to 
achieve optimum results 1n stress reduction, we 
have to work with both the chemo-iihyslcal 
stress factors and the proprioceptive Informa- 
tion. Only a coordinated effort on all areas 
can assure good results, since otherwise Iso- 
lated "peaks" of stress can affect the results 
of all the efforts. (It 1s oftenjnoted that an 
Isolated disturbance 1n an otherwise undls- 
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turbed environment will generally be much more 
noted and have more effect than if surrounded 
by a higher level of other disturbances..) 

In order to achieve a better,^ more ef- 
fective, and more operational interface by 
structuring the man-machine dialogue, we must 
have as a goal the reduction of stress. Stress 
itself is in some respects linked to informa- 
tion. There can be all kinds of information, 
not only those regarding the task itself. 
Stress, from this point' of view, can be caused 
by both under-information and over-informa- 
tion. If one gets too little information, one 
instinctively starts to worry and tries to fill 
the gap, searching for more information. This 
applies not only to information needed con- 
sciously for the task, but as well to subcon- 
sciously needed information regarding the en- 
vironment and oneself, e.g., .proprioceptive 
information. Over-information as well can pro- 
voke stress and thus hinder th* proper accom- 
plishment of a task. We must therefore strive 
to suppress all unnecessary and superfluous in- 
formation. Stress reduction by structuring the 
man-machine dialogue goes beyond the anthropo- 
metrical positioning of controls based on the 
criteria of reach and operability and beyond 
positioning of indicators, in order to read 
them better and more quickly. Structuring the 
man-machine interface is a very complex proc- 
ess which, under given time-cost constraints, 
and' manpower constraints, requires new, more 
effective tools and methods. 

THE ROLE OF MODELS IH SYSTEM DESIGN 

Many different tools have been developed 
that attempt to handle the complexity of param- 
eter interdependence in the anthropometrica I 
field. Man-models have been developed and suc- 
cessfully applied in many different areas. 
Techniques include bidimensional approaches, 
the simplest being the Dreyfuss scale, the 
Bosch drawing aids developed by Dr. Jemk of 
Darmstadt University, and other very special 
application-oriented approaches, like the com- 
puter-based design for car driver s worksta- 
tions by Volkswagen, the Reach model for astro- 
nauts used by NASA, and more complex man-models 
such as SAMMIE,- BOEMAN, CAR (Edwards, et al., 
1976), CAPE (Bittner, 1975), COMB I MAN (Evans, 
1972 Kroemer, 1973; Dillhoff, Evans, and 
Krause, 1974), and HOS (Strieb, 1974; Lane, 
Wherry, Strieb, 1977). It is tempting to con- 
sider developing even more complex computer- 
based man models, where many more parameters, 
like comfort and maximum limit* of movement and 
joints, forces, reaction time, manipulative 
precision, resonance frequencies of- organs, 
, etc., would be considered. The data from these 
models could be stored and automatically- 
studied in relationship to postures within the 
working environment. '* 



Such models are feasible and given the 
increasing power of modern computer systems 
will be developed as man reaches toward more 
perfectionistic levels of elaboration of such 
man models. However, because of economic con- 
siderations and pending the development of such 
elaborate models, we must be satisfied with 
less perfectionistic approaches and make opti- 
mum use of available low-cost computer-aided 
design systems, available specialized man- 
power, and last but not least common sense. 



COMPUTER-AIDED DESIGK SYSTEMS 

Today three-dimensional computer-aided 
design systems (3-D CAD systems) are available 
that, for a reasonable price, can run on the- 
medium-sized computers already available in 
giany institutions and companies. The Enhanced 
Interactive Design Systems (EIDS), a CAD soft- 
ware package that we use for engineering, 
architectural design, and human engineering of 
man-machine interfaces, can run even on small 
16 bit computers such as an HP1000. With such 
a CAD system, we have a tool at hand which can 
be used in developing interesting alternative 
approaches to workstation layout. 

A tool is, of course, only worthwhile if 
it is both helpful and cost effective. Al- 
though initially expensive, the cost effec- 
tiveness, i.e., the return on investment, of a 
3-D CAD system is favorable if you take into 
account that through its use we are really mul- 
tiplying the output achievablejfrom the rather 
limited human factors manpower 'avai lable in the 
system design field. The rate of return on 
investment is very much dependent on the use 
one makes of the data base once it is devel- 
oped. In our engineering designs, we make re- 
peated use of the data in designing (and rede- 
signing) the man-machine interface. Once we 
have established the linkages between the data, 
the engineers can continually redesign the 
"black boxes" and their contents, to optimize 
the system's functionality. 

State-of-the-art CAD systems allow a 
total three-dimensional description of objects 
and structures. For most crewstation design 
situations, only a true 3-D solid modeling CAD 
system will suffice. So-called 2^-D systems 
are not of much help^nor are wire-frame 3-D 
systems that lackPthe hidden line and hidden 
surface capabilities of. solid modeling sys- 
tems. The data structures of a 3-D solid 
modeling system are such that*the topological 
description of the r£al object is stored in the 
computer. All imaginable views, planes, iso- 
metric, or perspective, as well as cutaways are 
just "aspects" (in t'he Latin sense of the 
word). The computer does not "think" of the 
objects as planimetric projections. But any 
desired view can be obtained by choosing a 
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viewpoint, a direction of view and the type of 
view, and striking a few keys on 'the computer 
terminal's keyboard. 

* A further important feature 1s the abil- 
ity to remove or suppress hidden lines and hid- 
den surfaces. Only with such features can we 
evaluate visibility and parallax problems and 
obtain views of the objects that adequately re- 
flect the real product or structure being de- 
signed. With such a system, we can design 
every item of our workstation and assemble 
these items and structures in many different 
ways, trying -out alternatives and testing for 
reach and legibility for representative popu- 
lations. Since the system is a 3-D system and 
since the computer's internal description is an 
entirely three-dtmensional topological one, we 
areable to try different inclinations of 
patreTs, examine parallax problems from dif- 
ferent viewpoints, etc. (Eichweber, 1981) 

But besides enabling us to control every 
detail of the layout and enabling us to examine 
different anthropometric conditions and pos- 
tures for representative populations, EIDS 
aids us in the process of organizing the man- 
machine dialogue by enabling us to try out dif- 
ferent arrangements of indicators and con- 
trols. * We can design and test symbols and 
scales and develop new integrated solutions for * 
the man-machine dialogue. We can determine the 
best organization for readability. If the in- 
dicators move, we can test different positions 
and combinations of the respective symbols, 
even in their extremes. We can "build" our 
workstations and examine them from any view- 
point, test and discuss them with others. The 
tedious time-consuitting and costly process of 
building mock-ups cat) in many cases be almost 
totally omitted. We dan ensu£e-4tart^n most of 
our man-machine intert^cg^little or nothing 
will have to be changed as the result of trials 
even if we go directly to prototype construc- 
tion. 

For^ different workstations involving 
similar/tasks, we can design functional 
modules/, which, once they are stored in the 
data base, can be reused for different applica- 
tions. For standard enginefering analyses, we 
can use the geometric data in finite element 
stress analysis programs and for the numerical 
control of parts ^m^hining equipment. 

Three-dimensional data bases can also be 
used for simulation processes. Indicator sys- 
tems can, for example, be represented on dis- 
play screens, linked to simulators of the re- 
spective workstation or weapons system, and 
then be used together with appropriate controls 
to simulate the response of the system to the - 
actions of the operators. This can be helpful 
for instance in the layout and immediate, 
testing of crew stations for airplanes,, fire 
control systems, traffic controllers, etc. 



Finally, we can use the data base to ob- 
tain the documentation drawings needed for 
training handbooks, service manuals, and il- 
lustrated parts lists. Perspective drawings 
and exploded views are readily-available by- 
products of the same data base. If one con- 
siders the extensive amount of manpower and 
time it costs to obtain these with traditional 
drafting -techniques, it is obvious that the re- 
peated use. of the same 3-D data base multiplies 
its cost-effectiveness. 

At Eichweber, for instance, we have used 
EIDS for the engineering design of the newest 
generation of, our Tactical Laser Illuminating 
Shot Simulators (TALISSI) systems for directly 
aimed weapons. All mechanical parts in the 
system were entered into the data base. Moving 
parts are simulated in their various opera- 
tional positions. Perspective and exploded 
view drawings for documentation are also pre- 
pared viith EIDS. 



SUMMARY 

In oar work, we have found 3-D CAD sys- 
tems to be a vital tool in the proper human 
engineering of a modern weapons system. Never- 
theless, a tool is only worthwhile if it is^ 
applied well. Human engineering can only be % 
done well if all available knowledge and cri- 
teria are applied adequately. 
* 

To achieve this goal, we use interdisci- 
plinary teams and what we call the Value Design 
method (Eichweber, 1981). This method is based 
on Value Engineering and helps to get close to 
the optimum in both function and cost. (By the 
word "design" we mean to indicate that man and' 
man-related functional considerations are in- 
herent in the conceptual design.) ^his approach 
helps us to easily integrate specialists into 
an interdisciplinary, goal -oriented working 
team, with the rules of Value Engineering and 
Value Analysis serving as the "rules .of the 
game." 
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INTRODUCTION 

f 

' The ever-increasing sophistication and 
complexity of weapon systems and the atten- 
dant rise in costs are parallel trends in the 
devices being procured to support weapon sys- 
tems training. Compounding the situation is 
f the explosion in simulation technology which 
has led to development of sophisticated train- 
ing devices. Ofter; the devices have resulted 
from the technology because it is available* 
and not as a result of an analysis defining 
the specific training requirement. * 

In the development of a training device, 
the first step is an analysis of the train- 
ing requirements which the device will satis- 
fy. Over the years these analyses have been 
non-existent or when conducted have taken 
various forms with differing levels of accur- 
acy and thoroughness. All too often the 
analyses have been a cursory assessment of 
the existing or projected training require- 
ments. As a result the design and develop- 
ment of the devices have been left in the 
hands of engineering, computer, and software 
specialists^ .Their primary orientation and 
interest is the development of training hard- * 
ware,which operates in accordance with the 
approved specification. This does not, how- 
ever, necessarily mean that the^device satis- 
fies the training requirements., . 

The emphasis on the engineering develop- 
ment of a training device has developed by 
default (i.e., the lack of an analysis of the 
training requirements). The lack of partici-, 
pation by human factors and training analysis 
personnel in providing the necessary input 
Velated to the training requirements, the * 
instructor role, ease of operation, and in 
particular the, design of the Instructor/Oper- 
ator Station (IOS), has been a major contri- 
butor to the situation. 

As a result training devices have been 
delivered with IOSs which are complex and 
difficult to operate. For example, many IOSs 
have multiple data entry methods such as 
switches, light pens, numeric keyboards, 



alpha-numeric keyboards, multi-function 
switches, pages and pages of CRT information^ 
which must be called up before the specific/ 
item can be located, etc. Extensive formal 
training is required for the instructor/oper- 
ator to become proficient. Daily us£ is 
required to remain familiar with the opera- 
tion of the IOS, In the "real world" situa- 
tion the instructor/operator may or may not 
have had formal training. More likely he 
has received an abbreviated on-the-job train- 
ing program and does not use the device daily. 
Many instructors are, therefore, ngt quali- 
fied to administer training in the*devite. 
The result is that training quality, quantity, 
standardization, and efficiency, suffer. 

10S design is not just the physical lay- 
out of a work place. It also encompasses 
* consideration of instructional features, in- 
formation management, software capabilities, 
and many other varied factors. To properly 
inqorporate^the many factors into an effi- 
cient training-effective design requires a 
systematic analysis process'. It is« apparent 
tha^ many' IOSs, currently in use\have not 
evolved through such an analysis and, design v 
process^ - ' , 

The purpose of this" paper is^o summar- 
ise the prdW em. areas^ which have resulted 
from analysis and design shortcomings and to 
present design principles and concepts to^ 
overcome these deficiencies. The foafrffs" . 
on operational flight tra,iners^OFT). Many 
of the problems and* recommendations, however, 
•vdire applicable to a wide range of training 
devices. 

RBOBLEM AREAS v , ^ 

There are significant differences in. the 
design features of existing simulator IOSs. 
Some Vork better ttoan others; som£ have,. , 

-'SOphisti carted features, but do not work very 
wellj others are simple and work quite W6T1. 
Collectively, however,- there are weaknesses v 

4 which characterize OFT IOSs-. These weaknes- 
ses atte as follows: 



- Layout; labelling, coding., etc. do 
not optimize device operation and 
minimize instructor workload. 

- Steps required to access many CRT 
displays are time consuming and 
inefficient^ : ^ 

Delays in display access cause inef- 
ficiencies in training problem con- 
\ trol and monitoring. 

- Data entry methods are. confusing, 
.-redundant; and inefficient. 

- 'Many features and capabilities are 

not needed, not used, or difficult, 
to use. 

- Instructor" training does not ade- 
quately prepare instructors for their 
roles and responsibilities in using „ 
simulators. * .' ' 

- Instructor roles and responsibilities 
are not completely defined and 
implemented. * 

- Student training syllabi are poorly 
developed and used. Standardization, 
organization, efficiency, and qua I- 
fty of instruction are, therefore, 
impaired. * 

- Simulator capabilities are not keyed 
to the training requirements (i.e., 

' objectives) bf the training system. 

• in which it will be used. The 
simulators are,, therefore, not pro- 
perly' integrated into the -training 
system as an efficient; element 
- ' ' whfch fills a clearly-defined need. 

- Instructor handbooks aYeJiot properly 
t organized apd formatted to provide 

instructed assistance in operating 
the device in a traintng situation. 
They are massive informational 
vermes , »not training tools. 

When taken'togetheAtfte clear impression 
is conveyed that the tofal concept of design 
for the instructor is very loose l>i : e., 
the systematic principles of man-maplw ne . 
interface, human factors ,, and training tech- 
nology have not- been properly applied). The 
net 'result is that planning and design for 
operation of the device are secondary com- 
ponents of the total simulator development 
process. Actual operation of the device 



and training using- the device, therefore, . 
suffer. 

In many cases it is evident that a num- 
ber of the features and capabilities-were 
included, not because they were clearly re- 
quired to accomplish the desired level of 
trairiHng.-but because no one really knew . 
whether or not they should be there; so they 
were put in. This tendency toward overkill 
"has resulted from two* primary causes: \\) 
failure to perform the proper training analy- 
sis-efforts leading up to device design and 
(2) the explosion in. potential device capa- 
bilities resulting from technological ad- 
vances in the state-of-,the art. The former 
can be characterized as follows: "We're not 
. completely sure what we need' to teach in the 
device or how we should teach it, so we 11 
stick these characteristics in just to be 
,sure " The latter could go something like _ 
this- "We do not know just how we'll use 
these instructional features, but we 11 put 
them in. anyway. They' re neat." 'The overki 11 
increases the. reqUi rements for software and/ 
or hardware and escalates cost. 

The easy way is to allow simulator . 
technology to dictate and include everything 
that it permits and hope every contingency 
is included. The rationale sometimes, ex- % 
" pressed "we may need it someday or its 
nice to have", or "it doesn't cost anymore . 
This approach is the development of a sub- 
stitute aircraft and not .a training vehicle, 
with the* result that due to the complexity 
of the IOS, the instructor pilot has neither 
the skill, the time nor the inclination to 
utilize the capabilities of the simulator. 
Thus training capabilities of the simulator, 
are degraded. t 

' While the capabil ities .and features may 
be very desirable on paper, they tremendous li 
complicate 'device operations and increase 
costs: Complications result from the amount 
of information which must-be handled, the 
variety of data presented to the instructor 
the complexity -of the simulator control, "We 
amount of training required for instruction, 
and human' limitations. 

ASSUMPTIONS * 

The data on .IOS problem areas lead to a 
set of assumpttons which provide guide ines 
and a framework for the design principles and 
concepts." The assumptions are as follows. . 

- A specific simulator training 



syllabus will be developed as part of the 
.overall pi annmgf design, and fabrication of 
a device. The syllabus will result from a 
systemati^front end analysis in which the . 
simulator is treated ^as a component of an ' 
■ integrated training system. It will be ap- 
proved by the user and will "form the basis 
for implementation of training on the" com- 
pleted device. Admittedly this assumption 
does not always embody the way things are 
usually done. Rather it embodies the way 
simulator acquisitions should work and is a 
direction in which acquisition policies and' 
procedures are moving. A well -designed syl- 

^us is an important prereqtfisi te to good 
simtXsator design. 




.fining system and human engineering 
personnel will participate in the design of 
the IOS. It is further assumed that these 
personnel will have the skills required to 
develop and validate the training tasks and 
syllabus to convert the validated data into 
an efficient IOS station. The emphasis is 
that the IOS vyill be, keyed to the training 
requirements ahd situation,, not to technology, 
excess capabilities, £nd designer , whims". 1 

- Instructor pilots (IPs) will continue 
as simulator training instructors for the 
forseeable future, This assumption has boft 
positive and negative aspects. Positively, 
it means that tactically and with respect to 
flight skills, the instructors will be highly 
qualified. Jhey-wiU possess the ability to 
closely identify with student problems. .Neg 7 
atively, theymay not ba highly motivated and 
trained to perform the roles and responsibil- 
ities of simulator\instruetors. IOS design 
must, therefore, encompass instructor aid' 
. features which minimize instructor shortcom- ' 
ings and'which maximize their strengths! 

t ' 7 Training of IPs will not improve in 
the future. As Vioted previously, most in- 
structor training is insufficient to properly 
prepare them for their role in admirfistering, 
monitoring, and evaluating training exercises.' 
IOS hardware and software must, therefore, be- 
easily interpreted and usedi and n\Us% provide ' 
support to offset the_ effect of insufficient 
training. 

• - Simulator technology w in continue to 
advance in rapid strides. It will, therefore, 
be increasingly important for analysts and * 
designers to stay abreast of technological 
developments and to'closely-evaluate their \, 
applications to inditfrd.ua! simulator acqui- 
sitions. 



- Simulators will be used as, training 
vehicles and not. as substitutes for the air- s 
craft. They will, therefore, contain, only 
those attributes which contribute to training. 
The attributes will be determined through a 
systematic analysis of the -training require- 
ments* and the training system.- 

DESIGN PRINCIPLES 

The flight simulator design ancfopera* 
£ion weaknesses, and the assumptions form 
the basis for the IOS design principles. The 
design principles serve two primary purposes; 
(1) to provide a summary of what should be 
done to correct the weaknesses discussed 
previously and to implement the assumptions 
and (2) tp provider se^of guidelines which • 
will direct the development of IOS design-' 
recommendations. The design 'principles are 
as follows: 

Reduce the instructor/operator re- 
quirements for formal training by.' 
providing an instructor/operator in- 
structional aid system* at or near the 
. IOS. \his feature wi 1 1 npt or)Jy pro- 
vide training in the operation of the 
deyice but will actively assist the 
IP throughout^the training exercise. 
t 

Emphasize the use." erf automated train-* 
ing as the normal mode of -training. 

* 0 

-\ Reduce the instructor/operator work-' 
load by automating the ancillary 
tasks. 

• • • , 

- Reduce the number and' type of data 
entry methods at the IOS. 

- Use "touch" panels on CRTs and elec- 
tronic "touch pads" on the IOS con- * 

, sole as the primary data entry' 
methods. 

t - Design IOS layouts to* enhance opera-, * 
tion, interpretation, sequences of 
actions, etc! - . 

Eliminate the use of mul ti-fUnction * 
control s/switthes, 

% - Standardize the nomenclature used o$ 
. controls to reflect IP terminology/ 

- .Investigate thevusfe of cp'lor on CRT 
displays to highlight important 
points. *" r . ^ » 

; • • & 
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As discussed above, display location is 
standard. There is, therefore, no concern 
with where a given di^l ay will appear. Also 
displays which are candidates to replace a 

•given display which is beirig presented on a 
CRT are selectable using the touch pads on 
the CRT face. In most cases changing dis- 
plays will require a one-step touch. In i 
most other cases software design should ' 
minimize the number of decision pofnts (i.e., 

-Steps) the, instructor must handle. 

- Minimum steps to manipulate training 
exercises. Minimum steps^to manipulated 
closely related to minimum steps to change 
displays. In this case, however, rather 
than changing displays for monitoring pur- < 
poses only, the instructor is locating the 
display from which he will af.fect training 
problem control (e.g., locate and select a 
specific emergency, locate and # activate a 
specific navigational beacon, etcj. 

- Large selection of programmed exer- 
cises. One of the 'strengths of the proposed 
approach to IOS design and operation is a 
high-quality front-end analysis. Among 
other things, the analyses yields a realistic 
training device syllabus which is keyed to 
the training requirements and training situ- 
ation. The programmed exercises designed 

,from thfc syllabus are, therefore, essential 
parts of the total training systerfl and should 
be administered to, each student. The large 
selection allows consideration of student 
skills and progress, a variety of equally 
difficult exercises for different students, 
standardized training, programmed exercise 
capability for all training piases, and 
elimination of time-consuming set-up required 
in conventional free flight exercises. 

m 

- Large selection of initial condi- 
tions. Setting up initial conditions in the 
free flight mode on many simulators is time- 
consuming and may be nbn-standard. A large , 
set of initial conditions' gives instructors'. ' 
the flexibility they desire and improves use 
of time' and standardization. Selection of a 
given set of initial conditions wonrid Key 
ottfer responses from the IOS. For* example,v 
selecting initial conditions for an- aircraft* 
on the parking ramp may activate display of 
the*normal cockpit checklist, or selecting 
initial conditions for an aircraft in mar- 
shall may activate\the carrier approach 

display. 4 * * 

« » 

- IAS teaching and prompting. The IAS 
has 'two primary roles. The first is to 



te^fch instructors how To use the simulator. 
In this role r it is a CAI terminal used to 
present information on the operation of the 
IOS and on the procedures for conducting 
training exercises. The second^rile is to 
prompt instructors during set-upkand execu- 
tion % of training exercises. In $his role 
the IAS is a job aid which improves instruc- 
tor efficiency and standardization of* in- 
struction. * , 

\ ' 

CONCLUSION 

X A quote from the 2F114 (A-6 WST) in- 
structor handbook, is as follows: "This vol- 
ume, is designed to give'the instructor the * 
flexibility necessary to provide^each student 
with the complex and repetitive training he 
requires." This emphasis on flexibility has 
led to IOS designs which are needlessly com- 
plex and which actually hinder training. 

, This is not to say that flexibility 
should be eliminated: certainly not. De- 
grees of flexibility are necessary to meet « 
the variety ofneeds in a typical FRS train- 
ing situation. A major thesis of this paper, 
however, is that too much flexibility is a. 
hinderance. To balance flexiftjHyy with 
ability to meet realistic training require- 
ments", the IOS should be accurately program- 
med based ,on the results of a thorough 
front-end 'analysis. ^ The software concepts 
presented above provide the guidelines for 
this programming. The more structured * 
approach embodied in the analyses and result- 
ing software will help enflire that simulator 
training is standardized yet sufficiently 
flexible to meet training needs, 'readily 
manageable, and is easy for instructors to 
learn to administer^ 
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- Upgrade student and inl^ctor train- 
ing and operatinq materials. # - 

- Improve reliability and maintainabil- 
ity by use of touch controls, elimin- 
ating light pps, etc. 

♦ 

- Reduce cost by limiting IOS sapa- 
abilities to twining requirements. 

Each of the IOS design principle's is dis- m 
^cussed in the following paragraphs. 

INSTRUCTIONAL AID SYSTEM. To implement the 
.principle to reduce the need for formal i* 
training of IPs, it is proposed that a com- 
puter aided instructional (CAI) system be 
incorporated as an integral part of the IGS. 
This Instructional Aid System (IAS) w ill have 
the capability of presenting instructional 
programs to the IP for .operation of the de- 
vice; provide* the necessary instructions to* 
the IP in, the set-up, operation, and conduct 
of instructional exercises for a selected 
mode of operation; and cue the IP as neces- 
sary during a training exercise. 

As a pure instructional tool- it will be 
programmed to present instruction on basic 
operations of the device to include topics 
such 4S control locations and operations, 
display formats and interpretation, exercise 
control, and use of related training material 
(e.g., instructor guides and checklists). 
During set-up and conduct of exercises it 
will aid the instructors via prompts which 
guide them through the required operational 
steps. , For example, set-up on most devices 
requires the -instructor— to make a sequence 
of decisions which establish the character- 
istics of the exercise. The IAS would step 
instructors through this process', thus re- 
ducing instructor training requirements and 
reducing error rates.. During exercises the 
IAS would provide cues arid instructions to 
enhance instructor performance, decision 
making etc., as he monitors, controls, and 
evaluates. 

AUTOMATED OPERATIONS. The most sophisticated 
simulator,' which.will be efficient to use in 
achieving the training objectives and which 
wi|l simplify the instructor workload, allow- 
ing him to concentrate on instruction <versus 
operation, is the trainer which incorporates 
automated (programmed) training exercises? 
Tjje 'apprehension that automating training 
will be highly structured and thus rigid can 
be eliminated by astute planning in the 
development of a simulator training syllabus 
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and tfie specific training exercises. By us* 
ingf the analytical approach, the planning 
will consider all the variables and contin- 
gencies required for achieving the training 
objectives and, therefore provide all the.' 
required flexibility in simulator training* • 

t Use of an automated simulator instrOc- 
tional system, by its very nature, {fill pro- 
mote standardization. The training exercises 
are* the same for each trainee. The informa- 
tion presented to the trainee is consistent 
and in the correct format and the performance 
measurement parameters and scoring procedures 
are the same for each trained. Evaluation 
of the trainee performance is more objective 
and, therefore, more valid. Due to simpli- 
city of the design of the IOS, Instructor 
Pilot activity at the IOS is greatly reduced, 
* # allowing him to concentrate* on his instruc- 
tional role. 

The us^of autcimated instruction will 
require, a reorientation in the concept of 
simulator training. This requires the recog- 
nitiorj^that the simulator is a training ve- 
hicle and not a poor substitute for the air- 
plane. As a training vehicle.it should be 
responsive only to the determined training 
requirements as defined by specific training 
syllabi. 

The use of automated training exercises, 
specifically designed to provide training in 
achieving designated training objectives, 
will eliminate excessive trainer set-up time 
required in a "free flight" mode. In the 
"free flight" mode of operations in many* 
devices, the instructor must determine, se- 
lect and insert every parameter such as 
initial conditions, aircraft location, fly-to 
points, ground communications facilities, 
geographic displays, environmental factors, 
and others. A manual- set-up time of 15-30 
grinutes is not unusual; this situation not 
only is non-productive time for the trainee 
but deprives him of scheduled training. 

* - * 

It is recommended that automation go one 
step further. The>e wpuld be no "free flight' 
mode as currently defined. Free flight would 
be a rt cross between current free flight and 
automation (i.e., certain operations would 
be automated, and .instructor prompting would 
be provided). For example, the instructor, 
could not enter initial conditions on-line. 
Rather, he would always select from the pro- 
grammed set of initial conditions. To accom- 
modate this feature the set of initial condi- 
tions would 'be large and would be systematic 
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i initial conditions to begin 
iautomati cally branch the - 
|ned displays and instruc- 
ir actions. This basic 
selecti-ng those displays 
can be predicted ahead of 
•time, based on tffe syllabus and. type of 
training to be Conducted, would be carried 
throughout the frfce flight mode. 

IOS LAYOUT. At lirst sight the IOS for a 
sophisticated aircrew simulator may be 
overwhelming in its size and complexity. The 
numbers of switches, displays, keyboards, 
and other assorted, input/output features may 
be ominous to the potential simulator in-, 
structor. Even after the instructor has be- 
come familiar with operation of the device, 
there may still be a feeling of being 
overwhelmed. 

The IAS and emphasis on automation are 
features which will reduce the complexity 
of the IOS. Even with these features, how- 
ever, there will still be requirements for 
enough controls and displays to present a 
confusing operating' situation. In order to 
reduce confusion and increase the efficiency 
of operation, the IOS design must be develop- 
ed through .a systematic human factors analy- 
sis. The analysis is not just to meet' the 
requirements of MIL— STD 1472C (Human Engi- 
neering Design Criteria foj^ Military Systems, , 
Equipment, Facilities). Iris intended to i% 
yield the most efficient training/operating y 
environment possible for the instructors. ' 
Principles such as placement based on fre- 
quency, criticality, and sequence of use, 
must be used. 

Characteristics of the CRT displays 
must be. closely considered. Many of the dis- 
plays used in existing devices are cluttered 
and difficult to interpret. One salient 
point is the possibility of using color to 
highlight selected portions of displays. 

DATA ENTRY. Data entry in many devices, is 
needlessly complicated. For example, in 
some devices combinations of light pens, 
fixed function keys, and variable function 
keys are required interchangeably throughout 
an exercise. _ This mixing of modes increases 
the tirpe required to gain proficiency and the 
probability of error and confusion. 

To remedy this weakness it is recommend- 



ed that data entry modes be limited to a set 
of fixed function electronic touch pads on ' 
the IOS console and touch panels on the IOS 
* CRTs. The touch pads available on each ART 
display would be a function of the trailing 
situation as reflected on the £RT and would 
vary from display to display." 8asically 
the fixed function keys would provide simu- 
lator control. The CRT touch pads would 
* provide training exerci^sej/control . 

STANDARDIZED NOMENCLATURE. Operating inef- 
ficiencies may be caused by unclear, confus- 
ing, ambiguous, or unfamiliar terminology. 
To help remedy th4 problem, it is recommended 
that standardized terms be adopted and used. 
Stnce IPs are the primariy operators of the 
simulators, the terms used should be in 
"pilotese". Abbreviations should be avoided 
when feasible. Coding shpuld be minimized 
and when used should be used in a clearly 
ir\terpretable ^easily renumbered scheme. 

SUPPORTING MATERIALS. Although the student 
and instructor materials are no^rdirectly 
involved in* IOS'deSign, they do play an in->^ 
direct role in that they wjll be used in con- 
. junction with the IOS to carry out training 
exercises. They'should, therefore, be de- 
signed to^ prepare the students and instruc- 
tors >to carry out their roles during training 
exercises. 

The IAS is intended to take over some of 
the functions served by the instructor train- 
ing and operating materials. Adjunct written 
materials, however, will still' be required. 
These materials include the 'instructor hand- 
book, exercise guides, and checklists. As 
noted previously, the quality of these mater- 
ials for existing simulators, is generally 
inadequate. Their shortcomings contribute 
■to -simulator utilization problems. For emer- 
ging systems they must be upgraded. 
f 

RELIABILITY* AND MAINTAINABILITY. Reliability 
and maintainability wi'll be enhanced by the 
application of the^design principles, stated 
above, to the IOS wfcich in turn will b£ re- 
flected in the total simulator. The elimina- 
tion and/or reduction of switches and replace- 
ment with touch controls; elimination of com- 
plex keyboards and light pens will make the 
IOS more reliable and require less mainte- 
nance. 

C0Slj|| Reduction in cost for the IOS/simula- 
tor is a primary goal in the application of * 
the design principles. Eliminating the 
overkill capabilities inherent in current 

4 * 
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i simulators will result in lower procurement, 
v operatiag, and maintenance costs. 

DESIGN CONCEPT 

The design concept is based upon Con- 
sideration of -four data sources: 

- Instructor tasks: the roles and 
responsibilities of IPs in administering 
training exercises. For example, brief stu- 
dent, select mode of operation, initialize 
exercise, monitor trainee, evaluate per-> 
formance, etc. ^ <? 

- Training tasks: tasks the trainee, 
will practice in the device: The^are thre 

♦tasks from a typical Instructional System* 
^ Development task listing which are selected 
for simulator training. 

- Design principles discussed in tire 
previous section. 

- Technology resulting from an assess- 
ment of current and projected state-of-tfre- 

. art - ■ i 

The design concept consists of tv|£>j com- 
ponents: hardware and software. The com- 
ponents interact and are dependent one upon 
the other. A major issue in any design in- 
volving both hardware and software is, es- 
tablishing the proper functional balanjfe be- 
tween the two. In pointing out the cn^tinc- 
tion between hardware and software, trfere is 
an emphasis that IOS design is much more" than 
using good human* factors principles irjt deter- 
mining what the lOy should look like.^ It is 
a systematic process of* determining £ne, in- 
formation storage, retrieval, display, and 
manipulation requirements and implemerrting 
tfcese requirements in a combination of hard- 
ware and software which optimizes instructor 
performance. Major design concepts ajre dis- 
cussed in the following paragraphs. 

HARDWARE. The IOS hardware must be reliable, 
maintainable and easy to operate and yet 
must contain the components required for 
control, monitoring, and evaluation.' Major 
hardware components of the proposed IOS de- 
sign concept are as follows: 

- The main console. The main console 
is compact and is designed for a single in- 
structor/operator. All layout is consistent 
with good human factors design principles. 
Particular emphasis is on ease of use of 
controls, orientation of CRTs for ease of 
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display interpretation, and a functional 
work surface which accommodates instructor 
guides, checklists, etc. 

- Instructor Aid Station (IAS). The •* 
IAS is a separate small system with associ- 
ated controls. It is an integral part of 

the IOS and serves two functions: (1) pre- 
sent instruction on basic trainer operation 
and (2) prompt and guide instructors during 
the course of training exercises. It may 
also be used to display selected cockpit 
instruments (e.g., multi-function display, 
horizontal situation indicator, etc.). 

- Instructor's control panel. Th£ in- 
structor's control panel is the major "hard- 
wired" part of the IOS dedicated to simula- 
tor control. U consists primarily of fixed- 
function electronic touch pads. 

'SOFTWARE. In order to combine simplicity of 
hardware with complexity of weapons systems 
and training problems., it is essential that 
software be designed to enhance instructor 
performance. The software must be simple to 
manipulate, present information in easily 
used formats, and facilitate problem control 
and monitoring. Major features of the pro- 
posed software design concept to meet these 
goals are as follows: 

* - Display continuity. A given display 
is always presented on the same CRT. There 
is no switching of displays* between CRTs at 
the instructors discretion. ,When a display 
is selected or is called up automatically by 
the software, it always appears in the same 
place. 

- Standardized displays. Each tyf)e of 
display has a distinct, precisely prescribed 
format which is always used for that type of 
display. Formats are developed to enhance 
interpretation and use of the information 
presented. Display highlights are empha- 
sized by spacing, graphics, bold alpffenumer- 
ics, etc. It is Ntisjx recommended that color * 
be. used to highlight portions of the displays. 

- Automatic presentation of displays.' 
Maximum use is made of software selection of 
displays, so that minimum instructor inter- 
vention is required. During pre-programmed 
modes all displays are software selected. 
During free flight the amount of instructor 
display selection is minimized through keying 
displays<to trainee tasks. 

- Minimize. steps to change displays. 
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As discussed above, display location is 
standard. There is, therefore, no concern 
with where a given display will appear. Also 
displays which are candidates to replace a 

•given display which is berng presented on a 
CfiT are selectable using the touch pads on 
the CRT face* In most cases changing dis- 
plays will require a one-step touch. In . 
most other cases software design should ' 
minimize the number of decision points (i.e., 

"Steps) the % instructor must handle. 

- Minimum steps to manipulate training 
exercises. Minimum steps'to manipulated 
closely related to minimum steps to change 
displays. In this case, however, rather 
than changing displays for monitoring pur- / 
poses only, the instructor is locating the 
display from which he will affect training 
problem control (e.g., locate and select a 
specific emergency, locate and, activate a 
specific navigational beacon, etc.,). 

- Large "selection of programmed exer- 
cises. One of the 'Strengths of the proposed 
approach to IOS design and operation is a 
high-quality front-end analysis. Among 
other things, the analyses yields a realistic 
training device syllabus which is keyed to 
the training requirements and training situ- 
ation. The programmed exercises designed 

,from the syllabus are, therefore, essentia? 
parts of the total training systerfl and should 
be administered to each student. The large 
selection allows consideration of student 
skills and progress, a variety of equally 
difficult exercises for different students, 
standardized training, programmed exercise 
capability for all training phases, and 
elimination of time-consuming set-up required 
in conventional free flight exercises. 

- Large selection of initial condi- 
tions. Setting up initial conditions in the 
free flight mode on many simulators is time- 
consuming and may be nbn-star>dard. A large 
set of initial conditions' gives instructors', 
the flexibility they desire and improves use 
of time' and standardization. Selection of a 
given set of initial conditions would Key 
other responses from the IOS. For- example,v 
selecting initial conditions for an- aircraft* 
on the parking ramp may activate display of 
the*normal cockpit checklist, or selecting 
initial conditions for an aircraft in mar- 
shall may activate^the carrier approach 
display. 4 * * 

- IAS teaching and prompting. The IAS 
has 'two primary roles* The first is to 



te^th instructors how To use the simulator. 
In this role .it is a CAI terminal used to 
present information on the operation of the 
IOS and on the procedures for conducting 
training exercises. The second, rile 'is to 
prompt instructors during set-upkand execu- 
tion % of training exercises. In £his ro\e 
the IAS is a job aid which improves instruc- 
tor efficiency and standardization of- in- 
struction. * 

CONCLUSION 

* 

,A quote from the 2F114 (A-6 WST) in- 
structor handbook, is as follows: "This vol- 
ume, is designed to give the instructor the • 
flexibility necessary to provide,, each student 
with the complex and repetitive training he 
requires." This emphasis on flexibility has 
led to IOS designs which are needlessly com- 
plex and which actually hinder training. 

This is not to say that flexibility 
should be eliminated: certainly not. Oe- 
grees of flexibility are necessary to meet 
the variety of needs in a typical FRS train- 
ing situation. A major thesis of this paper, 
however, is that too much flexibility is a, 
hinderance. To balance flexittjjj/y with 
ability to meet realistic training require- 
ments', the IOS snpuld be accurately program- 
med based t on the results of a thorough 
front-end'analysis. ^ The software concepts 
presented above provide the guidelines for 
this programming. The more structured " 
approach embodied in the analyses and result- 
ing software will help enflire that simulator 
training is standardized yet sufficient^ 
flexible to meet training needs, readily 
manageable, and is easy for instructors to 
learn to administer* 

) • • 
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ABSTRACT 

ne of the issues facing the designer of sophisticated simulators today 
is how bo control their operation^ The traditional approach has been to 
supply tie instructor/operator with a series of hardware devices (push buttons, 
*eys, switches*, etc.) with which to command the machine. A^recent trend 
has move/ the functions to a CRT where commands are activated via an ^associated 
4-ight n^n, special 5unc>tidif keys , or, in some cases by touching the CRT screen. 
Although the methods'' of entering commands to a simulator are changing, the 
commands themselves are not. ^ f 

Tfcfi.8 paper,, explores a concept, in* which series of lower level events 
of commands are grouped into a larger entity called, for the sake of tfiis 
discussion, a "task module 11 . Manipulation of 'these task modules permits 
simpler operation of a simulator and the task modules themselves provide 
natural vehicles for performance measurement , scenario generation, brief ing/de*- 
briefing,* replay, and other 'associated functions. JLn the future this concept 
can provide the framework for modeling intelligent adversaries or supporting 
organizations in multi-unit simulators. *y 
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BACKGROUND . 



Simulators for flight* training purposes 
have been in use for many year*. As material 
J*nd energy costs climb, the need for inexpensive 
effective training has .correspondingly grown. 
The simulator is a natural tool for this 
kind of training/ The typical- s imu la tor 
of today is a highly mechanized collection 
of dials, gauges and displays, connected 
to a computer, relying on limited numerical' 
inputs by a skilled technician to drive 
a mocjcup of the equ'ipment ( being simulated. 
# A typical »example is the 2F95 F-14 Operational 
Flight Trainer (OFT). 

Typical Cockpit Simulator Characteristics/Op- 
eration t 

The F-14 Tomcat is a jbv S . Navy high 
performance aircraft. To fulfill its, role, 
the aircraft contains very sophisticated 
and complex avionics and weapon systems. 

The 2F95 OFT is a relatively modern 
deVice consisting of a' simulated F-14A cockpit' 
Cpilot only) mounted on a motion platform 
capable**tT£*-providing pitch, roll, heave, 
and lateral displacement about the related 
axes. Visual scene simulation i*s provided 
by a single channel, narrow field of view 

• • > - * • . 
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VITAL III-S display that provides night/dusk 
scenes for both land 'and carrier-based oper- 
ations* V> 

The 2F95( incorporates a single XEROX 
Sigma 5 computer system for all simulation 
and interaction with the student and instructor. 
Operation is controlled/^ rem a remotq instruc- 
tar/operator station located away from, 
but in sight of, the simulated cockpit. 

\ > 

The instructor controls operation 
in two ways, on4 sets up an exercise and 
one (controls operation during the training 

8€S8\On. 

the set up operation of the OFT is 
done by "programming" an Alphanumeric Data 
Display (ADD) -system. This system* consists 

, of a CRT terminal and a ten key numeric 
keypad. The ADD system has a t^otar*of 22 
informational displays, nine of which deal 
with the parameters of flight (that is, 

* carrier site, sea state, wind state and 
so forth).. To create an exercise, the infor- 
mation on thes£ displays must be updated 
to reflect the intent of the current training 
session. By editing the information on 
the parameter pages r the instructor programs 
the malf unct ionSi and reposition options 
to be encountexeo^in the training exercise. 
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ff ' The second method of controlling the 
exercise is via a special control panel, 
which contains hardware switches, dials, 
and indicators that allow direct manipulation 
of the simulator (fig. 1). From this panel 
the instructor can reposition the aircraft, 
insert malf ilnctions , ad just air turbulence 
and sea state a*n'd control the motion platform. 



Now the plane is on the runvay. 

4. To program the malfunction, a 
new page number must be entered to put up 
the malfunction display. 

5. An engine fire, represented by 
another code number must be adde^ to the 
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Figure 1 . OFT Control Panel 



Unfortunately, this method of control 
results in the instructor spending more 
time playing with numbers than with doing 
the primaTy job, instructing a student. 
For example, to give a student experience 
in dealing with an engine fire on takeoff, 
the following input* must occur. 



i following 
J If Fii 



First, the plane must be on the 
end of the runway to start takeoff. To 
do this, one must enter the ADD page number 
of the reposition display on the keypad 
mentioned previously.- 

2. Next, a code number representing 
the desired position is entered. 

4 3. Then a reset button is pressed. 



list of available malfunctions. Another 
number, associating this malfunction with 
a malfunction- insertion push button on the 
control panel, must be entered. 

6. When the student starts takeoff, 
the instructor monitors the airspeed indicator 
and inserts thfe malfunction by pushing the 
malfunction button at precisely the right 
time. 

7. /k fire light goes on in the cockpit. 

At no time during this button pushing 
exercise has the instructor accomplished 
anything that has a direct impact on teaching 
the student. In the worst case, the entire 
inst rue t iotial f unc t ion can be lost in the 
mechanics of controlling the simulator. 
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Instructor Support System 

v • • 

To alleviate this situation and othtr 
problems associated with simulator training, 
the U.S. Navy through NAVTRAEQU IPCEN sponsored 
the development an Instructor Support System 
*(ISS). The purpose of this system is to 
•take the mechanized aspects out of bhe training 
program and replace them with intelligible 
user interfaces that describe an exercise 
functionally and in terms familiar and acceptable 
to^any -trained flight instructor. " 

The ISS completely replaces the 2F95 
I OS with a new station consisting of two 
graphic CRT displays, placed one above the 
other, with a touch sensitive device overlaid 
on the lower display. All information needed 
to conduct a training session is presented 
dynamically on these displays and all control' 
of the problem and the simulator •itself 
is via the selection of menu choices with 
the touch panel. 

J . t 

, The subject of this paper is, but one - 
aspect of fhe ISS. Other important ojnes 
are dealt with in other papers presented 
at' this workshop* 

TASK MODULE CONCEPT 
* ♦ 

In a system as c-omplex as the F-j^4 
OFT a valid 'training structure is imperative . 
To facilitate such a structure, logic suggests 
the breakdown of the* student pilot's tasks^-^ 
into functional groups. For the sake of 
this discussion these groups are called 
"task modules' 1 . A task module is a logically 
related series of external events and pilot 
actions that should result in the meeting' 
of predefined criteria. Examples might 
be the takeoff from a particular airfield, 
a pre-stfcrt checklist, or a wing sweep mal- 
function. Task modules as conceived here 
have certain important characteristics and 
qualities. A brief identification follows, 
more detailed discussion is provided later. 

1. They' can be "run" or executed. 
That is, they have beginnings and endings 
in time." They can run in parallel, that 
is, two or more task modules can run at 
the same time independent of each other * 

.2. They can be easily identified 
and manipulated. This forms the basis of 
high level simulator control, exercise defini- 
tion, replay, and the like. 

3. Th6y provide^ the vehicle for 
associated training functions, such as perfor- 
mance evaluation and record keeping. 

* TASK MODULE TYPES 

In the development of this concept 
task modifies have quite naturally fallen 



into three, types or groups." They are. normal, 
flight and malfunction* Normal task modules 
include pie- and post-flight • checkouts and 
checklists dur»infe flight* Flight task modules 
, encompass those tasks related to flying 
skills, procedure* and navigation* Malfunction 
task modules relate to system failures; 

Normal 

^These are non-emergency procedures 
such as engine starts, checklists, etc. 
An example of this type is the takeoff 
checklist. In this case, the task module 
is activated by an instructor request at 
the touch panel. At this time, the checklist 
appears on the instructor's diiplay containing 
the items to J>e accomplished (fig. 2). 
Additionally, the training system may monitor 
these procedures. . The student pilot will 
perform such actions as are required, and 
when through^ indicates* completion verbally. 
To end this task mQdule, the instructor 
pushes .another button. The checklist disappears , 
and data concerning performance is recorded 
for later replay, review, or analysis. 



Flight 



Flight task modules are those concerned 
with the student's ability to carry out the 
mission, be it navigation, landing practice, 
or the Hke. This can be exemplified by 
the TACAN II approach to,Miramar Naval Air 
Station. The pilot flies a tear drop pattern 
from. an initial approach fix to a final* 
approach fix (fig. 3).- In this situation, 
the events beginning and ending the task 
module are entirely automatic. The instructor 
can devote a»l 1 energy to monitoring the 
student's performance. The task module 
starts when the pilot is going in a specific 
direction at a specified speed and altitude. 
When the student has started the approach, 
and by inference, the approach task module, 
monitoring begins again. This time, however, 
the events being monitored are those specifically^ 
related to the approach task. The student' 
flies at certain speeds, headings, and altitudes 
to follow the proper approach patterns. 
Whe"n the student completes the approach, 
'based on speed, altitude and heading, the 
task module also ends. 

Malfunction 

Malfunctions are activated by instructor 
request, again through the use of a push 
button* In this case, however, the button 
is on a touch panel, and is labeled with 
the name«of the malfunction to be inserted, 
for example , > ENGINE FIRE. Once* the instructor 
ha 8 prease'd the but ton , an ind icator light 
appears in the student cockpit and a malfunction 
chedklis't appears on the instructor's display 
(fig. 4). As the student follows procedures, 
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Figure 2. Takeoff Checklist 
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Figure 3. Tacan 2 Approach Checklist 
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Figure, 4. Malfunction Display 



updates appear on the checklist indicating 
activity in the dockpit as it occurs. If 
a student completes ail procedures correctly, 
the malfunction is removed automatically, 
or*, if the instructor has requested it, 
may stay in until the instructor chooses 
to remove it manually with another button 
press. 

APPLICATION OF THE CONCEPT 
♦ t\ 
Simulator Operati on 

In the* examples above the relationship 
'between task modules and simulator operation 
is implied. The key is ^je linking of the 
larger. un'its of the* training operation to 
the simulator directly. In the ISS this 
is done via touch selections of dynamic 
menu items put on the display. That is, 
the instructor is now allowed to manipulate 
the problem using tools that are natural 
to the application, not those tied to the 
simulator hardware, in a sense then, the 
task module structure becomes a kind of 
high cruder language, where pilot-relevant 
expressions replace numerical code commands. 

In contrast the procedure outlined 
earlier for inducing an engine* fire on* takeoff, 
a single menu selection marked ENGINE FIRE 
OiKTAKEOFF will accomplish the same thing. 



Exercise Definition 

Exercise building is much simplif iedU 
Here the task modules can be likened to 

x building blocks. A^cjomple'te training session 
can be developed quickly and_ conveniently 
"from a list of task modules. In the exercise 
set up facility of the ISS, ^groups of available 
task modules which are related in function 
are presented to the instructor in logical 
sequence (fig, 5). The/ 'instructor then 
simply selects the task modules which are 
appropriate for^the exercise. All such 

, building can be performed at the start of 
an exercise^ and thus the instructor does 
not have to work "on the fly" to make sure 
all exercise components are covered. 

Performance Measu rement 

r-^ ' - , 

Performance measurement and monitoring 
is also simplif ied Grading parameters^ 
and performance criteria may be associated 
with specific task modules. The computer 
is therefore free to monitor only those 
events which have direct bearing on the 
task module procedures being graded. Without 
such a mechanism to limit the processing 
requirements, the system designer is faced 
with the prospect of having to watch everything 
all v the time. This can be an overwhelming 
requirement. 
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Figure 5. Exercise Definition Selection Page 



A related J>ene f it in this area is 
the self limiting nature of the performance 
data generated. Diagnostic messages and 
grades sre 0I J-y displayed for task modules 
associated with that training session. 
This simplifies and reduces the amount of 
data the instructor needs* to examine to 
make valid decisions regarding the student's 
performance and abilities. 



Modeling Outside Agencies 

~k 

Another area in which task modules can improve 
a training session is handling mathematical 
models of outside agencies. Usually the 
actions of FAA controllers and other individuals 
who interact with the pilot can be linked 
to a Single task module. For example, all 
the actj-vity of a GCA controller can be 
linked to a final approach task module.. 
Thus, a specific function can be tied to 
a specific task module. This contributes 
to a* simplification of software design. 
If a function changes, the related task 
module can be altered without interfering 
with programs or other task modules. Modulari/y 
is *a natural part of the system. 

An extension of this concept, which 
has not been incorporated into the f SS , 
is that, task modules could be built to provide 



intelligent adversaries . or supporting entities 
in multi-role simulators. For example, 
the skill or aggressiveness of an enemy 
pi-lot in an intercept ^trainer could be easily 
manipulated using the task^module concept. 
In another application the imaginary* crew 
of an ASW aircraft could be commanded to 
perform some complex manuevers in support 
of a shipboard ASW simulator via a simple 
menu choice. 

Debrief /Replay/Record Keeping . * 

*♦ . , j s 

Debriefing the student is much enhanced 
by the use of task modules. The student 
can be shown a ipeci'fic tasl^ during a replay 
without extraneous information unrelated 
to the learning experience. Since grading, 
is linked td a task module, record keeping 
becomes more viable .and more clear to_student 
and instructor alikev — Aay-^task^ module may 
be replayed in any order, saving^ time and 
enhancing tj^e student 1 s- under s t~aTidi4Xg_bv_ 
emphasizing the specific areas that need 
work. This is JLn sharp contrast to *the 
problems that ensue when, replay is time 
related and th<e action of interest must 
be searched for, or worse', when no replay 
is available. 
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STANDARDIZATION AND FLEXIBILITY 

The concept of handling operation 
via high-level task modules also hat some 
secondary benefits. • ' . x 

Generally, the task module "also provides 
< standardization, which allovs'student performance 
to be> ev^U^ed more consistently. ' A task 
module wil^a/lways start at the same time 
under the same conditions. The same procedures 
must be followed ani nothing will be overlooked. 

At first glance i.% may appear that 
this approach may reduce general flexibility * 
in simulator operation." This could be a 
problem if only a, small number of task modules 
are maintained ,^ However, to support a full 
syllabus such as that associated with F-14 , 
% j replacement training, a iarge number (150 r 

in the ISS)' is required. This, -plus the 
* fact that the task modules can be linked 
in a variety of ways, assures adequate f lexibi- 
bility. 

SOME CAUTIONS , - . 

In order for this concept to work *v 
effectively the task modules must be constructed 
very carefully. Less than a very thorough 
analysis of start/stop events can cause 
task modules to run or stop running unexpectedly. 

Haphazard definition of scoring criteria 
can have very detrimental effect of user 
acceptance of the system. 



'The greatest problem faced by tie 
developers of the 185 was. an ^nderesfcimat ion 

« of the resdurces required to define the 
training tasks frqmVbich the task modules 
* * were derived an4-W£Lime and effort required 
to create J*nd checifctJut the task modules.. 
A feans h'as^een identified tb^ease the 
latter burden By* automating aome of the 
task module *creat ion functions. This will 
'enable members of the user's community ti> 
create and modify task modules interactively 
at a computer terminal. This will not require 
pxogramming skills nor a knowledge of the 
simulator itself, but it will require a 
thorough and ve^y detailed understanding 
of the learning process, not a trivial task. 
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INTRODUCTION, 

Recent- advances in Information processing and 
display capabilities have permitted significant 
increases in both the number and -the complexity 
of training features found an today's simulator 
> instructional systems. These advances reflect 
a combination of more detailed user requirements 
as well as the rapidly increasing complexities , 
of the systems and vehicles being simulated. , 
' * "* 

These oveha>l advances in capabilities have ' 
produced a corresponding and steady increase . * 
in system -operating and control requirements, 
ffforts to advance the state of the art have 
focused primarily on system capabilities and 
fidelity rather than instructional value, 
oftfcn short-changing the role and the needs 
of the simulator instructor. These ancj other 
factors have contributed to increasing task 
loading up to and in some cases beyond accept- 
able levels. This need not be the case; in- 
structional system control and operation- can 



be made \simple. and straight-forward*. Jask 
loading can be dramatically reduced and 
instructional effectiveness increased through 
careftfl front-end actiJHties combined with 
the systematic application of appropriate * 
design principles. These principles concern 
the recognition of the goals of the training 
program and the unique capabilities and re-r . 
quicements of the simulator instructor in 
meeting these goalsr 

The simulator instructor, like the flight 
instructor, performs several key functions 
in the instructional process. Some of these 
functions can be automated while some can be 
fefcil ltd ted .through the provision and 
formatting of training-relevant information. 
In -addition, information and control capa- 
bilities } can be made available with minimum 
instructor intervention. The instructor's 
worklQad can be minimized and, equally 
significant, the instructor can be given ready 
access* to the instructional capabilities he 
requires. 



. THE' INSTRUCTIONAL SYSTEM. 

The function of the simulator instructional 
system 15 to support the goals of the instruc- 
tional program and to support the instructor 
as ne performs the tasks required in facilitat- 
ing instruction. TJie instructional system can 
be optimized through the application of two key 
design considerations: OPER ABILITY and ACCEPTA- 
BILITY. Operability is achieved when an in- ' 
structional system is as easy to use as is 
practical, while acceptability is achieved when 
the system makes optimum use of the instructor's 
unique talents and of the simulator's unique* 
capabilities. Wljile these are somewhat .simpl i- 
fied and generic definit-Tbns of criteria whicfi 
touch 6n virtually every aspect of system design, 
they do address the mam concerns. For thjs 
reason they' are the primary guidelines for the 
system design approach to be developed here. 

Except for the aircraft pref light, the jobs of 
the simulator instructor and the flight instruc- 
tor are closely analogous. They perform highly 
similar functions, including the arrangement of 
training conditions, briefing, demonstration, per- 
formance monitoring and jdiagnosis, modification 
of preplanned training exercises, coaching and 
guidance, performance evaluation and critique, 
and communications functions. In addition, of 
course, the flight instructor devotes a major 
part of his attention to acting as a safety 
pilot. The flight instructor performs these 



tasks by observing the performance of the 
student and the aircraft, comparing these 
.performances with the prescribed performance 
stapdards and with his y6wn estimate of the 
capabilities expected of the student $t a 
gaven time. In addition, he facilitates learn- 
ing by prescribing practice conditions .within 
the limits .lipposed by. the flight- environment, 
v 

TJ&e simulator has the inherent capability of 
providing more task-relevant information than 
is available to the instructor in flight. In, 
addition,, U has the inherent capacity for pro- 
viding and standardizing practice conditions as 
they are required for effective instruction, and 
it. has the ability to provide information in many 
different forms and relationships. ' 



AN* EXAMPLE. 



v 



An example 'of a training exercise as it might 
occur in a typical simulator is used to illus- 
trate the application of twelve basic design 
objectives used to achieve optimum instructional 
system operability and acceptability, \hese 
design objectives assume that the simulator 
and its instructional program ^jave been developed 
in response to a set cff well-defined trairftng 
objectives. The definition- of these objectives 
results in the identification of relevant practice 
conditions, performance measures, and performance 
criteria reflecting , student progress ir) the • * 



training exercise. The design objectives are as 
follows: ! * • 

\ v Minimize the number of individual^ controls 
x " and functions on ttie instructor console 

2. "Automate non-instructional tasks to the 

greatest extent possible 

3. Minimize the instructor control a'ctions 
$ required t(y perform all operating and 

control, tasks 

4. Minimize -requirements for changing displays 
during critical Instructional periods; 

• * . anticipate and program displays needed in 
each'pha^e of the exercise „ 

5. Provide maximum instructor flexibility in 
the control of the instructional process 

6'. Provide- continuous trainee performance 
feedback 

7. 4 Correlate the simulator's graphic display 

and performance' data capabilities with 
trainee monitoring techniques used in 
4 the aircraft - 

8. ' Minimize requirements to enter or modi fy 

variable data through exercise pre-planning 

9., Provide instructor selectable performance 

data recording " 

y 

\10. Make maximum use of relevant advanced 

instructional techniques to enhance feed- 
back, control the training" setting, and 
r simplify performance data interpretation 

X 

11. Minimize use of the instructor for exercise 
preparation 

12. Design eaCh feature to minimize requirements 
for handbook reference 

Many of these guidelines appear to be contra- 
dictory. How, for example, does one design an 
instructional system to provide maximum instructor 
flexibility and control while restricting the 
number of controls, control actions and* modi-: 
ficatiohs of variable data? 

The answer lies in a thorough analysis of the 
training-to be provided in the simulator. The 
training analysis must define the training ob- 
jectives >to be-addressed, the optimum conditions 
necessary for effective training, the perfor- 
rpance parameters to be monitored, and the cri- 
teria associated with acceptable performance 
and learning. In .addition, the training - 
analysis must.'ctnticip<*te the needs of both the 
student and tife instructor in practicing and 
-learning- and ;fiCdirecting and instructing.* The 
functional requirements *1n the training analysis 
are reflected 1n the approach illustrated in 
the following example as they indicate antici- 
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Figure 1. Training Management System Overview 

pations of straining events and instructor 
invol veurent. . \ 

The example usfed here assumes the existence 
of a- system by which academic, simulator, and 
flight training are managed. Figure 1 is an 
overview of a typical training management system.- 
Data used in scheduVing training resources -and 
aircraft and simulator maintenance as well as 
data relating* ttf syllabus preparation and stu- 
dent grading are handled by the training 
management system. Data are entered through 
computer .terminals in appropriate areas of the 
training facility. 

The instructional design approach illustrated 
in this example is concerned with three major 
phases of a typ-ical. simulator training session. 
The first phase includes the activities of the 
instructor from the onset of his preparation . 
tcTsks up to the point of commencing- the train- 
ing session." The second illustrates instructor 
conduct of an instructional task of moderate 
complexityv while the final phase covers post- 
exercis'e activities from the completion of the 
final twining task to the completion of the 
debrief. 

Each of ,these three phases begins with a dis- 
cussion of instructor activities performed for . 
a training flight in the aircraft, followed by 
the corresponding activities and supporting 
instructional system characteristics required 
for a simulator traiQing session; A one-to-one 
correlation is used to simplify the presentation. 
The training tasks, task sequences, and grading ( 
methods and criteria are representative of today s 
instructional environment and its requirements. , 
'"Single Engine Landings" has been selected as 
the typical instructional task. An instructional 
environment relating to a mid-phase aircraft, 
familiarization^ session is used to provider 
overall continuity. 

TRATMTMfi PREPARATION - AIRCRAFT. 



Prior to commencing the briefing the instructor 
obt&ins a print-out of the flight grading form 
by way of the remote flight Training Management 
(JMS) terminal located in the briefing area. 
Thfc fonn provides an up-.to-date list of the 
required training for this flight since it is 
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Figure 2. Representative Grading Form 

an output of the TMS training selection process 
resulting from the request for data. The 
instructor then obtains the complete training 
record file which includes all 'previously com- 
pleted and annotated flight grading forms and 
reviews this information to identify performance 
trends, weaknesses, and 'strengths. External 
fa.ctors such as weather, traffic density and 
restrictions, and practice landing fieldrse- 
lection are introduced to complete the in- 
structor's overall flight planning requirements. 

The formal* briefing is a fairly standard in- 
structional' event and reqtrires no special con- 
sideration here. Typically, it includes dis- v 
cussions and review of the flight profile or plkn, 
sequence of events, environmental factors, per- 
formance expectations, procedural knowledge, 
and 6ther trainee fl.ight preparation require- 
ments, and other related areas as applicable. 
On completion of the briefing, the instructor 



~Wnd trainee proceed to the flight -line, 'review 
the assigned aircraft maintenance data by way 
of the remote aircraft maintenance TMS terminal, 
and then perform the aircraft preflight con- 
cluding the preparations for flight. 

TRAINING PREPARATION - SIMULATOR. ^ 7 

Instructor preparations for the corresponding 
(mid-phase familiarization) simulator session 
/are identical .to those required for the aircraft 
' excepting those areas that pertain solely to the 
actual flight (i.e., external environmental 
factors, air traffic instructions, aircraft pre- 
flight). The grading form is obtained by the 
. instructor through the remote simulation TMS 
.terminal located tn the simulator briefing area. 
The format, identical for both flight and simu- 
lator applications, is shdwn in Figure 2. 

As an output of the TMS the form provides 
'the instructor with supporting information ■ 
tnat is not available through conventional 
means. Note, for example, .that a performance 
indicator (dris identified for each of the 
listed training areas or tasks preceded by the 
letters P or R. This indicator represents a 
past performance average accumulated from all 
previous instruction. The letter P denotes 
tasks or performance areas primarily introduced 
and graded while R identifies areas of difficulty 
where performance has been substandard and addi r 
tional (beyond the norm) review and instruction 
is required. Tasks preceded by the letter I are 
to be introduced for the first time, while A 
identifies areas of advance instruction to be 
introduced providing time and trainee performance 
permit. 



Upon completion of the briefing, the instructor 
and trainee proceed to the simulator, pausing 
briefly at the remote simulator maintenance TMS 
terminal to review simulator and instructional, 
system status. Upon arrival at the instructor 
console the START function is selected on thp 
instructional system input device configured 
as showrt in Figure 3. 
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Figure 3. Instructor Input Device 



This actioft initializes all resources of the 
Instructional system in preparation for the 
activation of the training selection process. 
To simplify the instructor's input procedures 
all displays and repeaters are blanked and the 
* following instruction is provided. 

ENTER TRAINEE IDENTIFIER 

The four digits selected on the numeric pad 
(Figure 3, extreme right) are repeated* for 
verification on khe CRT immediately beneath 
the displayed instruction. Assuming the 
Selection is acceptable to the TMS,( incorrect 
or unacceptable entries are ignored and specific 
correction instructions provided) ENTER causes 
the displayed data to be cleared and the following 
data and instruction- presented: 
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The phase is identified through selection of 
the corresponding enclosed numeric, in this 
case 1, for the familiarization phase in the 
instructor's input device (Figure Z,, lower 
lef£) Selection again causes all display . , 
data to be cleared, and through direct inter-; 
4 face with TMS, initiates the training selection 
process. After a short pause (during which an IN- 
PROGRESS message is provided), the following in- 
struction and corresponding control function are 
displayed: 

j2 INITIALIZE FOR TRAINING 

Selection causes a complete training initial i 
zation to occur by positioning and configuring 
the simulated aircraft, selecting and depicting 
the appropriate visual scene content to both the 
instructor and trainee, and displaying the appli- 
cable monitoring and control data for thp initial 
task at the instructor console. Upon completion of 
these processes, the simulator is placed in freeze, 
and a READY message is displayed to the instructor. 

In review, the simulator preparation procedures 
have included: 

1. Selection of tye START function to perform 

a complete instructional system reset. 

*2. ENTERing the trainee four-dtgit identifier. 

3. ^ Selecting the desired instructional phase. 

4. Activating the training intial ization process. 

These procedures differ significantly from 
those used with systems that employ extensive 
and "time-consuming selectipn and modification 
of instructional- data in the special exercise 



preparation or planninq^mode. Mary of these 
systems extend simulator turnaround times beyond 
acceptable limits and often require extensive 
instructor knowledge of complex and cumbersome 
data manipulation procedures. 

INSTRUCTIONAL TASK-AIRCRAFT. 

The flight has logically and sequentially pro- 
gressed on through to the pattern entry, and the 
practice of previously introduced normal landings 
has been concluded. As an introduction to the next 
maneuver the instructor takes control of tfce air- . 
craft and demonstrates a single-engine down- 
wind and approach to a touch-and-go landing 
(simulated By retarding of throttle to idle). 
Extensive verbal commentary is provided on 
cockpit procedures and aircraft handling tech- 
niques including the shut-down procedures tor 
simulated engine fire on down-wind. In addition, 
the instructor must initiate and respond to 
required voice communications with the tower 
and monitor the traffic pattern 'to insure safe 
aircraft separation at all times. 

• * 

Upon completion of the demonstration the in- 
structor returns the aircraft to its normal 
two-engine pattern configuration, and once re- 
established .on downwind,- control is returned 
to the trainee. 

The instructor then introduces the maneuver by 
announcing a simulated engine fire and begins 
to monitor the trainee's procedural response and 
control of the aircraft, coaching and guiding as 
necessary. In this capacity the instructor s 
sequence of monitoring activities might typically 
include: 



1 Procedural response to the simulated engine 
fire as related verbally by the trainee; 
the trainee reduces the throttle to idle 
at the appropriate time to simulate the 
engine shutdown 

l 2. -Response of the trainee to the aircraft 

flight requirements (e.g., the addition .of. 
power on the remaining engine to maintain 
altitude and airspeed, maintaining balanced 
flight, smooth basic airwork, etc.) 

3. Establishing the proper downwind path for 
the existing wind condition and arriving 
at an optimum single engine abeam distance 

4. Simulated transmission to the tower requesting 
emergency clearance 

5. Completion of landing checks and trans- 
mission to the tower requesting landing 
clearance insures gear check is verified 
and landing clearance is received and 
acknowledged 

6 Basic, aircraft control during the approach, 
focusing on. angle of bank, irate of descent, 
angle of attack, power setting, verifies 
parameters at key checkpoints 
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Final approach for proper distance at roll- 
out, sroopth basic airwork, proper alignment, 
minor corrective adjustments 

Touchdown in first third of the runway on 
center line at the optimum rate of descent 
and angle of attack r 

Acceleration on the -runway for aircraft 
control on center line, and smooth power 
application 

Optimum pitch angle »t lift-off; stable 
lateral control, smooth acceleration to 
single-engine climb speed, and a positive 
rate of climb throughout to pattern 
altitude 



11. 



Proper angle of bank during downwind 
to establish an optimum 
distance from the runway 



turn 



In addition, and throughout the maneuver, in- 
struction is provided, the traffic pattern is 
•monitored for safe separation, all transmissions 
on the selected frequency are monitored, and 
trainee performance- is noted for post-flight 
analysis, debriefing, and grading. The maneuver 
is repeated as required to achieve the desired 
level' of skills development for this flight. " 

INSTRUCTIONAL TASK-SIMULATOR. 

The corresponding simulator'training" session 
has also logically and sequentially progressed 
on through all the pattern entry, and ttie prac- 
tice of previously introduced normal landings 
has been concluded. Upon instructor request, 
the instructional system is advanced to the 



next task area (SINGLE-ENGINE LANHtf&SJ as the 
trainee completes the crosswind tfirn tb the 
.downwind leg. Through this simple request 
(discussed later) the required instructional 
system features (visufl repeater, CRT displays, < 
and' supporting programs) are selected and 
configured to monitor and control this specific 
training event as it evolves* (in addition to a 
view of the visual scene of interest, the 
instructor is provided with two CRT displays 
configured as shown in Figure 4). 

The primary display (on the ]eft) provides a 
graphic depiction and tabular performance list- 
ing in the main areas, with supporting data *5nd 
controls to the right. The secondary CRT display 
(on the right) provides the instructor with a 
view of the applicable cockpit instruments and 
indicators with supporting problem and task 
controls depicted to the ri^ht. The arrangement 
of the instruments and indicators* corresponds 
to the rear aircraft cockpit configuration to 
support the normal in-f light scan patterns of 
the instructor. 

Figure 4 also identifies the applicable control 
functions from the instructor 'input device 
(Figure 3) used to interact with, each display 
type. Since the functional arrangements are 
dedicated, the requirement to assign controls 
to a CRT and the accompanying input errors that 
often result from such an arrangement have 
been eliminated. * > 

Continuing wjth the task, the instructor takes 
control of the siipulated aircraft by activating 
freeze (Figure , : 3, upper right) arrf alerts the 
trainee to an Impending initialization. As an 
introduction to the maneuver the instructor 
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Figure 4. Task Display Configuration 
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selects function 6 by way of the input device 
(Figure 4) to access the prerecorded demonstra- 
tion. When selected, the supporting data 
area of the primary display is updated as shown 
in Fiqure 5. Function 1 is selected, causing 
the simulated aircraft to INITIALIZE for the 
start of the demonstration. Upon completion, 
a READY indication is provided (lower area) 
with the system awaiting the release of freeze 
?o initiate the'playback. Function 2 is seized 
to activate the accompanying prerecorded verbal 
commentary. > 

-Upon freeze release, the instructional system 
takes control of the simulated aircraft and 
demonstrates a prerecorded single-engine down- 
-vund approach to a touch-and-go landing W so 
simulated by retarding the throttle to idle). 
The simulated single-engine configuration is 
again used to maintain continuity with the pre- 
viously descnbeTaircraft training flight 
'maneuvers. Extensive prerecorded verbal com- 
mentary is provided on cockpit procedures and 
aircraft handling techniques, including the 
shutdown procedures for a single-engine fire 
on down-wind. Commum cations wifh the tower 
are included in the commentary. 

As the simulated aircraft passes the 90-degree 
position in the approach, the instructor momen- 
tarily deactivates the verbal commentary to pro- 
vide additional remarks that are applicable to 
this specific trainee based on past performance 
trends.* At touchdown the commentary is reacti- 
vated. * 

Throughout the demonstrated maneuver, the CRT 
displays and visual repeater provide all normal 
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feedback to the instructor, including the per- 
formance data (discussed later) depicted n the 
lower mam area of the primary display (Figure 4). 

Once the simulated aircraft returns to the down- 
wind leg and while still under demonstration 
control, a normal flight configuration is re- 
stored and the primary display is cleared of 
all accumulated track and performance history. 
AX this point, the instructor a erts the trainee 
and returns control of the simulated aircraft 
to him (Figure 5 - Function^) and then resets 
the supporting data area to its original data 
configuration (Function 6) Once assured that 
the trainee is in comfortable control of the 
s tuat on the instructor introduces the maneuver 
oy announcing a simulated engine fire and begms 
to monitor the trainee's procedural response 
and control of the simulated aircraft, coaching 
and' guiding as necessary. In this capacity, the 
instructor's sequence of monitoring activities 
and supporting system control inputs might 
typically include: 



1. 



3.. 



4. 



Figure^. Demonstration Control 



Procedural response to simulated engine 
fire as related verbally by the trainee 
throuqh the simulator communications system. 
The instructor verifies, the the correct 
throttle is reduced to idle at the appropriate 
time by way of the ACTION MONITOR feature 
(pnmary display) -or the THROTTLE position 
s?atus (secondary display). No control or 
selection inputs are required. 

Response of the trainee to the immediate 
aircraft flight requirements (e.g., tne - 
addition of power on the^remaimng engine 
to maintain altitude and airspeed, main- 
taining balanced flight, smooth basic air- 
work, etc.) by scanning the cockpit instru- 
ments and indicators by way of the secondary 
display in the forward view scene of interest 
in the visual repeater. No control or selection 
inputs are required. 

Establishing the proper downwind 
for the existing wind conditions and 
arriving at an optimum single-engine abeam 
position by observing the aircraft symbol 
and track history depicted relative to the 
optimum dowrwind corridor (primary display, 
upper area) and abeam position (key check- 
point B) and by scanning the abeam view 
area of interest on the visual repeater. 
No controller selection inputs are re- 
quired. 

Simulated trainee transmissions to tower 
requesting emergency clearance with the 
simulated communications system. Frequency 
is verified on the ACTION MONITOR feature 
(primary display) and the instructor re-, 
sponds to the transmission or selects a 
pre-recorded message. Aval 1 able me ssages 
are accessed on the PROBLEM CONTROL INDEX 
(secondary display, Function 11). The 
desired message (Figure 6) is activated 
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scene of interest; no control or selection 
inputs are required. 
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fiol RESET DISPLAY* 



Figure 6. Message Control 

- upon selection of the appropriate function 
— in tjiis case, 12. , 

Completion of landing xhecks by observing 
the ACTION MONITOR feature and the cockpit 
.instruments arrd indicators display. .Trans- 
mission to the tower requesting landing 
clearance is monitored in the simulator 
communications system and the instructor 
transmits or "activates landing clearance 
(Figure 6) and monitors for acknowledgment 
and gear. check verification from the 
traineg. 

Basic aircraft control during the approach 
focifsing on angle of bank, rate of descent, 
angle of attack, and power setting as pro- 
vided by the cockpit instruments and indi- 
cators (secondary display); verifies parame- 
ters at key checkpoints by observing the 
aircraft track history and performance data 
areas of the primary display reinforced by 
the varying scene-of-interest view from the 
visual repeater. No control or selection 
inputs are required. 

Final approach for proper distance and 
runway alignment at roll-out by observing 
the aircraft symbol relative to the Ideal 
distance (checkpoint 6) and the optimum 
path reinforced by the visual scene of 
interest. Basic airwork and minor correc- 
tive techniques are observed in the cockpit 
Instruments and indicators display and are 
also reinforced by the view of the visual 



Touchdown on the first third «f t\f runway 
as indicated on the runway depiction (pri- 
mary display) on center line as viewed from 
the visual scene of interest and at the op- 
timum attitude rate'of descent, etc. as 
depicted in the qprfomtance area of the 
primary display (checkpoint TD), no control 
or selection inputs ara required. 

Acceleration on the runway for center-line 
control as monitored in the visual scene* 
of interest and smooth power application 
depicted in the THROTTLES status area of 
the secondary display; no control or 
selection inputs are required. 

Optimum lift-off attitude (performance 
data area, checkpoint LO), smooth acceler- 
ation to single-engine'climb speejL and a 
positive rate of climb to level <nf^at 
pattern altitude as observed on the 
cockpit instruments and reinforced in 
ijfle visual scene of interest. The in- 
structor transmits the downwind clear.- 
ance or activates the applicable pre- 
recorded gtessage. j 

Adjustments to angle of bank during the . 
downwind turn taarnve'at an optimum 
distance 'abeam the runway as depicted by 
the aircraft track history on the primary 
display, supported -by the cockpit instru- 
ments and the visual sceYie of interest for 
b^sic aircraft control and airwork. No 
Control or selection inputs are required. , 
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Upon completion thej[)rimary data display pro- 
vides the instructor with a concise, graphic 
and alphanumeric performance history of the 
maneuver. When combined with occasional notes 
on basic airwork and general performance trends 
recorded as required by the instructor* 4 com- 
prehensive base of informatioa is made available 
for post-training analysis debrief and perfor- 
mance reassessment. A sample primary data 
display depicting a completed touch-and-go 
is shown in Figure 7. (Some of the depicted 
data is applicable to additional instructional 
capabilities to be addressed shortly.) 

The HARDCOPY feature (Figure 4, Function 2) 
provides a printed copy of the entire display 
as it appears on the CRT and is commanded by 
the instructor at the conclusion of each 
maneuver. The print-out (which does not affect 
the optfrability of the system) occurs when se- 
lectedV thus avoiding storage problems and the 
del ays Yf ten encountered when processing large 
amounts V accumulated data at the conclusion 
of trainyng. Once a print-out has been commanded, 
the track history and performance data may be 
cleared (Figure 4, Functions 3 and 4) at the 
instructor's option 1n preparation for the next 
maneuver: 
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Figure 7. Sample Primary Display Data 

Note that a number of additional simulation 
capabilities are provided to the instructor; 
these offer training assistance, and ta*5k control 
not available during aircraft instruction. 
The supporting data area of the secondary dis- 
play provides access to a variety of these man- 
euver-applicable control features (Figure 4, 
PROBLEM CONTROL INDEX). In addition to the 
prerecorded messages discussed previously, 
environmental controls and engine failures 
are also available for selection. 

The index includes control of environmental 
conditio/fs and- parameters that are required 
in providing the flexibility for complete 
instruction. Changing the wind cpmponent, 
for example, requires the trainee to alter 
his approach path over the ground, while 
varying the temperature affects engine perfor- 
mance and resultant power requirements^ In- 
creasing 'the turbulence level makes smcrotff baSi£ ! 
airwork more difficult, while limiting visi- , 
bility effects judgments .which depend on the 
pilot's perception of the fli'ght area. Varying- 
aircraft upright and center of gravity can 
totally change handling characteristics requir- 
ing altogether different approach techniques. 
Since the simulator provides the only safe 
environment for the development of proficiency 
in sjngle-engine landings, the index also pro- 
vides access to selected failures to realis- 
tically create the required conditions. 

All selection and control functions are accom- 
plished in the supporting data area allowing 
, the cockpit instruments and indicators to be 
^continuously displayed for performance in-status 
morvitoring. All control actions (e.g., selection 
of desired malfunction, wind componerft, air- 
craft //eighty etc.) are accomplished through 
us« of the enclosed, numeric functions (Ref- 
erence 7, Message Activation). The seTerctelT 



conditions ,are dis^fryed and the lower graphic 
area of the prin-ary^i splay as shown in'Figtire . 
7. Active malfunction titles are depicted in > 
the lower left and the letter M is displayed 
1n the graphic maneuver area at the poiot of 
activation. This information provides the 
instructor with a continuous -display of simu- 
lator status during training and also facili- 
tates posl-training maneuver reconstruction 
and debrifefing. 

The IWITIALIZEfiVunction, also found in the 
supporting dataaFea immediately ab^ove the 
PROBLEM CONTROL INOEX, resets 'the simulated ' 
, aircraft-to an optimum position from which 
tjie task may be commenced. As a visual 
aid to the instructor, the position is 
identified in a graphic area (Figure 7) by 
depicting the corresponding function. Control 
and system checks are perfonned^upon selection 
and appropriate cuing messages provided to 
insure the simulated aircraft is compatible 
with ttje demanded configuration prior to re- 
lease of cdntrol to the trainee. 

The., ACTION MONITOR system depicted in the 
supporting data area of the primary, display 
(Figure 4) is a continuously active procedure 
monitoring display feature that lists all 
trainee actions that are detectable by the 
computational system. The CLEAR MONITOR 
function provides the instructor with a means 
of erasing all displayed actions* in preparing 
'to monitor a specific procedureor sequence. 
As a part of the primary display, ACTION 
MONITOR data is -included in all HARDCOPY print- 
out selections'. 

Interaction with* the final instructional system 
feature assigned for use with this task and to be 
addressed here is also accomplished with the 
primary display. Selection of the depicted 
function (Figure 4) reconfigures tflfe. supporting 
data area with typical R£SET and REPLAY control 
features as shown irt Figure 8. As a visual aid 
to the instructor irc selecting the optimum tirne^ 
increment, positions corresponding to the available 
reset points>are depicted in the graphic area of 
the primary display on the simulated aircraft 
, track history (Figure 7, Diamond Symbols). 



These basic and straightforward control and 
monitoring features offer th£ instructor all 
of the flexibility required to successfully 
complete the prescribed instruction with mini- * 
mum distraction and maximum commonality with 
aircraft training procedOres. y 

Obviously, the simulator provides much greater 
capability for the control of training in many 
significant respects than the aircraft. By the 
same token, however, these capabilities must be 
implemented in the simulator and in the instruc- 
tional system with operability and acceptability 
as primary guidelines. These are reflected in 
the example used here as unique simulator capa- 
bilities are. organized to^support the methods 
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Figure 9. Representative Grading Form 

simulator TMS terminal and integrated with the 
existing data in the trainee history file. The 
training- selection process is initiated and the § 
grading form for the next simulator familiari- 
zation training session is printed out. Its 
contents and corresponding preparation require- 
ments are briefly reviewed with the trainee 
(who retains the form for subsequent reference). 
The instructional event is concluded by placing 
the completed grading form in the trainee's folder 

DRAWING THE CONCLUSIONS. 
f ' * 

Although brief and limited to somewhat con- 
fined areas, the information presented "should 
provide a sufficient basis to allow the main 
issues identified at the onset to be re-addressed. 

Have the two key design considerations of 
operability and acceptability been satisfied? 
Would the design approach promote ease of use? 
Is it one to which instructors could relate? 
There is a third and equally important consi- 
deration as well: Is it realistically achievable? 

How well did the design. approach conform to 
the twelve objectives and guidelines stated at 
tlie onset? Specifically: . 

a 1. Were the number of individual controls 
* and functions minimized? 

2. Where required instructor control 
actions minimized? 

3. Did the design minimize requirements 
to enter or modify variable data? 

4. Were non-instructional tasks auto- 
mated to the greatest extent possible? 

5. Were display switching requirements 
minimized? 



Was maximum instructor flexibility 
and control of the problen provided? 
Did the system prGvide continuous 
feedback of trainee performance? N 
Was performance data recording 
instructor selectable? 
Did graphic and performance data 
correlate wi£h aircraft monitoring 
techniques and procedures? * 
Were applicable advanced instructional 
features made available? 
Did exercise preparation minimize 
the use of the simulator? 
Would ^it be possible for new instructors 
to operate. this system totally without 
referencing a handbook? 

One remaining issue needs to be resolvetJ be- 
fore any valid conclusion can be drawn. How 
were these guidelines and objectives selected, 
and do they accurately ^identify the critical 
Resign areas to be addressed? Since they have 
aHsfand for some, repeatedly) appeared in 
InilitSN^. procurement specifications in the 
recent past, it is fairly safe to assume 
that they. do. 

SOME FINAL THOUGHTS. 



Effective and efficient simulator instructional 
systems can be designed and produced today, but 
a few significant trends must^f altered. First 
is the specification that caws for minimum in- 
structor task loading, ,but then demands maximum 
system capability combined with complete and 
continuously available instructor control of 
all system variables and training features. 
Minimum task loading is normally stated as 
a design goal and maximum control as a require- 
ment. Since the two are essentially incompat- 
ible when specified in this manner, the require- 
ment must obviously take precedence, often re- 
sulting in systems that are characterized by 
unnecessary and burdensome operations. This 

/Situation is further compounded by the erron- . 

'eous tendency to measure capability and train- 
ing effectiveness by the amount of instructor 
control provided. These and other similar 
trends serve only to mask system capabilities 
and discourage effective utilization. 

-The answer, therefore, will not necessarily be 
found in more systems and more technology, but 
more likely from a willingness on the part of 
tjoth users and designers to make the true dis- 
tinction between want and pfeed when identifying 
the trainee, instructor and the system require- 
ments. This is. a matter of analytical procedure 
tempered by self-discipline — a process that 
must be totally Oriented toward the specific 
learning objectives of each system. 

THE WRITER'S CONCLUSION. 

This paper has explored and expanded on some 
very basic design concepts emphasizing practical 
application and common sense. It has offered no 
really new or innovative technology, but simply 
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Figure 9. Representative Grading Form 
\ 

simulator TMS terminal and integrated with the 
existing data in the trainee history file. The 
training- selection process is initiated and the. 
grading form for the next simulator familiari- 
zation training session is printed out. Its 
contents and corresponding preparation require- 
ments are briefly reviewed with the trainee 
(who retains the form for subsequent reference). 
The instructional event is concluded by placing 
the completed grading form in the trainee's folder 

DRAWING THE CONCLUSIONS. 

Although brief and limited to somewhat con- 
fined areas, the 'i nformation presented 'should 
.provide a sufficient basis to allow the main 
issues identified at the onset to be re-addressed. 

Have the two key design considerations of 
operability and acceptability been satisfied? 
Would the design approach promote ease of use? 
Is it one to which instructors could relate? 
There is a third and equally important consi- 
deration as well: Is it realistically achievable? 

How well did the design. approach conform to 
the twelve objectives and guidelines stated at 
trie onset? Specifically: . 

i 1. Were the number of individual controls 
* and functions minimized? 

2. Where required instructor control 
actions minimized? 

3. Did the design minimize requirements 
to enter or modify variable data? 

4. Were non-instructional tasks auto- 
mated to the greatest extent possible? 

5. Were display switching requirements 
minimized? 



6. Was maximum instructor flexibility 
and control of the problen provided? 
V- Did .the system prGvide continuous 

feedback of trainee performance? N 

8. Was performance data recording 
S instructor selectable? 

9. Oid graphic and performance data 
correlate wi^h aircraft monitoring 
techniques and procedures? 

10. Were applicable advanced instructional 
features made available? 

11. Did exercise preparation minimize 
the use of .the simulator? 

12. Would *it be possible for new instructors 
to operate. this system totally without 
referencing a handbook? 

« 

One remaining issue needs to be resolvetl be- 
fore any valid conclusion can be drawn. How 
were these guidelines and objectives selected, 
and do they accurately ^identify the critical 
Resign areas to be addressed? Since they have 
alSsjand for some, repeatedly) appeared in 
InilvbNcy. procurement specifications in the 
recent past, it is fairly safe to assume 
that they. do. 

SOME FINAL THOUGHTS. 



Effective and efficient simulator instructional 
systems can be designed and produced today, but 
a few significant trends must*j£ altered. First 
is the specification that caws for minimum in- 
structor task loading, ,but then demands maximum 
system capability combined with complete and 
continuously available instructor control of 
all system variables and training features. 
Minimum task loading is normally stated as 
, a design goal and maximum control as a require- 
ment. Since the two are essentially incompat- 
ible when specified in this manner, the require- 
ment must obviously take precedence, often re- 
sulting in systems that are characterized by 
unnecessary and burdensome operations. This 

situation is further compounded by the erron- . 

'eous tendency to measure capability and train- 
ing effectiveness by the amount of instructor 
control provided. These and other similar 
trends serve only to mask, system capabilities 
and discourage effective utilization. 

The answer, therefore, will not necessarily be 
found in more systems and more technology, but 
more likely from a willingness on the part of 
tjoth users and designers to make the true dis- 
tinction between want and pteed when identifying 
the trainee, instructor and the system require- 
ments. This is. a matter of analytical procedure 
tempered by self-discipline — a process that 
must be totally oriented toward the specific 
learning objectives of each system. 

THE WRITER'S CONCLUSION. 

This paper has explored and expanded on some 
very basic design concepts emphasizing practical 
application and common sense. It has offered no 
really new or innovative technology, but simply 



9 

ERLC 



124" 



> 



1 ' -< 



proposed some logical and straightforward 
approaches 1n helping to resolve some rather 
old, frustrating, and costly problenfc. It 1s 
not the answer, but 1f your response to some 



or even just a few of the areas addressed was 
positive, then perhaps it is a step 1n the 
right direction and merits further consider- 
ation. 



r 



* 



125/126 



THE INSTRUCTIONAL SUBSYSTEM, A MAJOR COMPONENT 
OF THE TRAINING DEVICE 



Dr. Thomas J. Hammell 
Eclectech Associates, Incorporated 
North Stonington Professional Center t 
North Stonington, Connecticut 06359 



ABSTRACT: 



> The instructional subsystem has tradition- 
ally been a part of every simulator/ 
training device. However, it has « often 
been a misnomer>A:onsisting of little more 
than an operator's station for the 
controlling^ of the sophisticated 
simulator/^ The necessary technology 
currently exists, both in the form of 
hardware/software capabilities and 
graining/instructional techniques,, for the 

^development and application of potentially 
rrtgh cost effective instructional tools to 
enhance the instructional process. Thus, a 
"bonafide" instructional subsystem should 
be a part of every training device. This 
paper addresses the methodology for 
developing the instructional subsystem, 
suggests* several capabilities of that 
subsystem, and presents an' applied example 
of an instructional subsystem in the form 
of SMARTTS. 



INTRODUCTION 

•The General Accounting Office, in their 
report to Congress concerning "How to 
Improve the Effectiveness of U.S. Fonces 
Through Improved Weapon System Design" 
(1981), focused on the importance of the 
operator to the overall effective 
functioning of the weapon system, and the 
current situation of insufficient early 
planning to provid^ an adequate operator 
interface. They estimated that human 
errors account for at least 50 percent of 
the failures of major weapon systems. They 
farther subdivided these failures into 
operator skill level and proficiency 
limitations, amongst other factors. ' Their 
findings attribute these problems to the 
increasing complexity of modern day 
systems. Two important issues are evident 
from this recent 'investigation. The most 
obvious is the need for improved training, 
as one means of improving operator 
proficiency and reducing operator error. 
This points to the need for more effective 
training systems, of which the instruc- 
tional subsystem is one contributing 
aspect, .albeit 3 major aspect. Second, the 
compjexity of training systems is 
increasing in parallel with that of the 



*> operational systems, requiring better human 
* factors 'design of the training system 
itself (i.e., instructor and trainee 
interfaces). Both of these issues point to 
the need for effective human factors design 
of the training system from an' instruc- 
tional process standpoint. 

This problem of developing a more effective 
training system is not- really new, since 
the training community has always been 
concerned with effective instruction. The 
predominant .emphasis in the design of 
training systems.^n recent years, however, 
has been placed on the engineering aspects, 
such as concern over adequate simulation 
fidelity and the technology to achieve that 
fidelity. Although cost-effective fidelity 
should be a major issue, proportionate 
emphasis should be given to the other 
training-related aspects of the training 
. device and system. In an investigation of 
the training effectiveness differences' 
attributable to differinq levels of 
fidelity of major simulation characteris- 
tics (e.g., 120 degrees versus 240 degrees 
visual scene horizontal field of view, for 
a shiphandling/shipbridge simulator), 
instructor differences were found to have 
several times the impact on the 
effectiveness of < training in comparison 
with any of the simulation characteristics 
investigated (Hammell, Gynther, Gra§so, and 
Gaffney, 1981). The relative importance -of 
the instructor was not unexpected .(e.g. , 
Caro, 197^), although its strength was 
surprising fh the presence of simulation 
characteristics selected for their 
potential impact on training effective- 
ness. The importance of this finding is 
that the instructor typically embodies most 
of those non-simulation characteristics of 
the training device and training system 
(e.g., exercise design, monitoring of 
student performance, student feedback). 
The instructor is that "catch-all " that 
magica'lly transforms the simulator into a 
training device. This, of course, should 
not be the case. The training device 
should, be more than just a simulator, it 
should, have capabilities specifically 
designed to augment the training process. 

The training device is rapidly becoming a 
most important element of the training 
system, and is oft.en the* . centerpiece of 



that, training system.; This paper focuses 
on the simulator-based training system, 
wherein the simulator/training device is a 
major element of "that system. Other 
• elements include the training objectives, 
the* training* syllabus and other training 
media (e.g., at-sea- training). The 
training device^as *t has often been 
traditionally known, is nothing more than a 
simulator. It ' merely^ seeks to imitate 
aspects of the real woHd, such as 
providing a-*radar display showing informa- 
tion similar to that which would be seen on 
the actual ship. The simulator is, of 
course, limited, in that which it can 
reproduce faithfully. Those aspects ,that 
a'rje ' simulated are presumably those deemed" 
necessary to the conduct of an effective 
trailing process, while many others are 
simply not addressed by the simulator. 

/^The training- device is more than a 
simulator" {Hammell, 1981). Whereas the 
simulator simply has a simulation system, 
the, training device, has' both simulation and 
training subsystems. The simulation 
subsystem is that noted above. The 
training subsystem, on the other hand, 
•should consist of capabilities that are 
designed* specifically to enhance, the 
training process via aiding the instructor, 
providing information to the_students, and 
so op. The instructional subsystem (i.e., 
training subsystem) has traditionally been 
a part of every simulator/training device. 
However, it has often been a misnomer 
consisting of little more ,than an operator 
'station - for the controlling of the 
sophisticated simulator; it has seldom 
really provided capabilities to support the 
training process. The necessary technology 
currently exists, both in the form of 

" hardware/software capabilities and 
training/instructional techniques, for the 
development and application of potentially 
high cost effective instructional features 
to enhance the instructional process. 
Thus, a "bon^fide" instructional subsystem 
should be a part of evecy training device. 

' The instructional subsystem should be , 
tailored based on the many considerations 
surrounding k the particular training 
application , J and the training device. 
Training assistance, capabilities should be 
provided to support rthe instructor, the 
trainee(s), and training system management 
(see Figure 1). The instructor support 
capabilities should provide tools and 
information for (1) development of traimnq 
exercises and support material^, (2) 
monitoring and control of the training 
process, and (3) assisting in achievement 
of an effective training process interface 
with the trainee(s). The trainee support 
capabilities are often coincident with f 
"(3) M . Management support capabilities 



should deal with inter and intra site 
coordination, the continual improvement # jn 

^training cost/effectiveness, and so on. 

' 'Although management support- capabilities 
are a part of the .instructional subsystem 
since they "directly impact the effective- 
ness of training, they will be only . 
cursorily addressed herein. * . J 

The availability of sophisticated 
computer-display technology and its 
traditional incorporation .in virtually all 
sophisticated simulator-based training 
systems enables a* wide variety of training 
assistance technology to be 
integrated into the simul ator-tfaSed 
training device. The issue, then, is 
twofold: (1 ) instructional technology 
should be incorporated as a major part of 
every training deVibe, and (2) the extent 
to which the instructional technology 
shduld be incorporated depends upon the 
• particular training objectives, - the 
capabilities feasibly available on the 
training device, and the many other 'issues 
and constraints surrounding th$ particular 
training situation. ' 



Third Generation Training System * 

The issues currently faced in, the 
development and integration of instruc- 
tional support capabil ities represent the 
start of the 'third generation training 
device. The first generation training 
device, which is typified by early efforts 
at simulation (e.g., Link Trainer) had *s. 
the primary cohcern simulation fidel ity. 
The problem at that time could be viewed as 
one of simulation fidelity at almost any 
cost. It was simply necessary to achieve a 
sufficient amount of fidelity in the early 
simulators so as to achieve what was 
considered as meaningful training. The 
primary requisite methodologies and 
research information at that time concerned 
engineering-related issues; that is, 
engineering design and simulation 
techniques. These issues, of course, still 
remain and will continue as relevant issues 
for the develbpment of simulators. 

As adequate simulation 'capabilities were 
achieved in several areas, the second 
generation of training devices emerged. 
This generation was no longer concerned 
with achieving adequate simulation at any 
cost, since the engineering capabil ities 
were often available. Rather, the second 
generation training device had m as its 
primary issue cost effectiveness ~+ the 
cost of* design with regard to the 
effectiveness of resultant training. 
Rather than maximizing simulation fidelity 
at any cost, the major considerations 
focu$ed on^ie minimal level of simulation 



fidelity necessary to achieve requisite 
training effectiveness (i*.e., identifica- 
tion of the essentia] features of the 
-simulator, and their respective levels of 
fidelity, required to meet specific 
training objectives * and performance 
standards). The variety of methodologies 
developed to address this issue focused on 
a systematic approach to the design of the 
training system and/or the training 
device. Examples include the Training 
Situation Analysis (TSA) 'and the 
Instructional Systems Development approach 
( ISD)-r^ These methodologies typically 
identified the specific behavioral tasks 
(i.e., including skills, , knowledge* and 
training objectives) of the trainee, and 
then sought to identify the minimum 
necessary simulator/training device 
characteristics for tfoeir achievement. A% 
was the^ case with the" first generation 
training device issues, the second 
generation issue (i.e., cost effectiveness) 
remains today, and will continue to remain 
pertinent. 

The training industry is' currently 
embarking on the third generation training 
device. As the second generation was a 
further refinement of the first, so the 
third generation training device and issues 
represent a further refinement of the 
second and first generations. Whereas the 
second generation's emphasis was on 
identifying the minimally acceptable level 
of simulation fidelity and thus minimizing 
the cost while maintaining effectiveness, 
the third generation will ♦ focus on 
providing training enhancement features to 
greatly improve the effectiveness *of 
training at relatively small additional 
cost. This further evolution of the 
training device will augment the earlier 
generations in substantially improving the 
cost effectiveness of training, by 
specifically , designing in instructional 
support capabilities. 

Recognition of the importance of the 
instructional support capabilities has been 
a fundamental problem, but one that is 
being steadily overcome in both the 
training and operational communities.. The 
major technological problem is twofold: 
(1) the development of appropriate 
methodologies and resulting research 
information pertaining to the design of the 
instructional subsystem, * and (2) 
utilization of these methodologies and 
information to' actually design the 
instructional subsystem. The third 
generation training device does not, as 
yet, have- appropriate methodologies 
adequately developed, much t less an . 
empirical information base fronr which to 
draw, to guide the design of the instruc- 
tional support subsystem. This ' is a 



substantial problem currently facing the 
training device industry^ 

Methodologies for . design of the 
instructional subsystem have not been 
formalized. Hence, the instructional 
subsystem has traditionally not been 
included as 6 part of the training device. 
Instructional subsystems, of course, have 
■ been developed and implemented on certa'in 
devices, although to an extremely limited 
extent. The instructional subsystem should 
consist 'Of (1) instructor support 
capabilities, (2) trainee interface 
capabilities, and (3) training system 
management capabilities. The remainder of 
this paper will identify aspects of these 
caRabil ities, and issues to be considered 
1n their development. Since one of the 
most effective means of instructing is that 
of presenting examples to the students, 
this paper: presents an example of an 
instructional subsystem known as SMARTTS 
(Submarine Advanced Reactive Tactical 
Training System), which has been recently 
implemented on a Submarine Combat System 
Trainer (SCST). 



INSTRUCTIONAL SUBSYSTEM CHARACTERISTICS AND 
ISSUES 

The instructional subsystem, as noted 
above, should address the instructor, the 
trainee, and system management (Figure 1). 
Each of these are considered as important 
elements of the instructional subsystem 
(Hammell and Crosby, 1980; Hammell, 1981). 
This paper, focuses particularly on 
capabilities to support tKe instructor and 
trainee. 

The Submarine Advanced Reactive Tactical 
Training System (SMARTTS) (Hammell and 
Crosby, 1980) is a pioneering effort in the 
development of instructional ' support 
capabilities, in that it is currently 
integrating a sophisticated instructional 
support subsystem with an existing 
submarine combat systems trainer. This 
project developed a substantial body of 
information regarding the * design and 
application of the instructional subsystem, 
and may' be used as a departure point for 
directly addressing many relevant issues. 
. The SMARTTS project has, for example, (lj 
developed and applied some of the 
potentjal ly useful techniques for the 
generation of information to design 
instructional support .characteristics, (2) 
has designed a variety of instructional 
support characteristics which are currently 
being implemented, and (3) is planning to 
embark on a test and evaluation of the 
instructional support subsystem and its 
particular set of , characteristics. The 
problems identified, the experiences, and 
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the techniques developed during the SMARTTS 
project provide considerable insight into, 
many specific issues- relevant to the 
instructional subsystem. 

The instructional subsystem is addressed 
below in terms of (1) design methodology, 
which discusses the overall approach to the 
design of th0 'instructional subsystem; and 
(2) training assistance technology, the 
specific capabilities^ of the instructional 
support subsystem,- % Several ' issues are 
addressed^ under each of these, with 
examples drawn from * SMARTTS as appro- 
priate. 'Also, concepts for future 
development are introduced. t 



Design Methodology 

Highly structured methodologies are 
available fQr the selection and design of 
training media (e.g., Training Situation 
Analysis (TSA), Chenzoff, 1965; 
Instructional System Development (ISO), 
NAVEOTRA 106A). . These methods, which have 
been demonstrated as effective, are 
primarily intended to» address the 
Student/trainee interface with the training 
media. For example, a major use of the ISD 
approach is to determine simulator/training 
device fidelity characteristics ,.(e.<g., 
level of visual scene fidelity). These 
systematic approaches relate the .training 
device/training system characteristics to 
specific trainee operational tasks. This 
of .course is necessary. However, such, an 
approach does not directly address those 
training device/training systems ' capabil- 
ities that can directly augment ; ,the 
training process but are not directly 
linked to the operational tasks (e.g., 
external feedback). These other 
training-related characteristics do not 
^derive from the operational tasks, but 
rather derive from the base of generic 
training research and methodologies. There 
,is obviously a need to design the training 
/systepi yith regard to both operational 
tasks* and generic training methodologies. 
In practice, both are 'ad<£essed, although 
the latter is not done' so formally. 



INSTRUCTOR INTERFACE ANALYSIS . • The 
available methods . tyave not adequately 
addressed the instructor interface (i.e., 
including various forms of instructor aids) 
t<x the training device/training system. 
Relatively little emphasis has typically 
been placed on the design characteristics 
of the training device/training systeij to 
, directly assist the instructor in 
\conducting the training process. Needed 
is a structured approach for the desigrr of- 
the instructional subsystem. The analysis 
Should be* quite similar to that of ISD, 



although not focusing on operationaf tasks, 
but rather focusing orv instructor taste and 
training methodological principles. *his 
type of analysis has been employed' in the 
past, - although to f very limited extent 
(e.g., the SMARTTS ^example presented later; 
"N^also, Charles, 1978). Several • elements , of 
an * ISD process for the instructional 
subsystem might include the following 



elements: 



\ ; 

analysis — a 
of the. functions 
performed by the 
aspects associated* 
role. . This 
the basis ' for 



Instructor task 
detailed analysis 
and v 'tasks to be 
* instructed on aTl 
with ,his training 
analysis should form 
the 'design^ of, training 'system/- 
traimng device characteristics to^ 
support the instructor in conducting 
an* effective training process. 

Analysis of instructor loading an 
estimate should be made of the load 
placed on the instructor in the 
performance of -his functions and 
tasks. This load should address for 
each task the difficulty, time to 
perform, frequency, ^ available 
resources, accuracy of < performance 
required, and so on. The intention 
is to- develop an accurate profile of, 
the loading placed on the instructor 
across his functions . and tasks, and 
his expected performance level on 
the basis V the training system 
design. 

Identification of ; critical 

instructor functions and tasks — 
based on the above two elements, 
those instructor functions and tasks 
that < would < benefit substantially 
from other assistance should be 
identified. For example, data 
recording is a task that a good 
- instructor -would perform frequently, 
so as to have good* postexercise 
feedbatk information. " Data 
recording can be extremely cumber- 
some and time cpnsuming; further^ 
more", it is a task tharcan often be 
readilyrperformed by the computer/- 
training "device, _thus 'off loading 
the instructor to devote more of his, 
time to those tasks that only he pan 
adequately perform (e.g., monitoring 
student communication). 

Determine instructor .support 
Capabilities — "specific instructor 
support capabi 1 i ties shpjutd be 
determined via^ a .cost/effective 
trade-off analysis >based' on the 
above elements, alternative design 
approaches for providing support 
capabi Titles, and the ir expected 
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effectiveness In assisting the 
instructor and enhancing the 
effectiveness,, of the training 
process. A-tnedia selection approach 
to support the instructor functions 
and r tasks should be performed, 
similar to that accomplished, under 
ISD for the trainee. - 

This type of *fropt-end analysis is devoted 
• to the instructor, and should result in the 
•generation. r of cost/effective training, 
device/training system characteristics. 
Although aimed at specifically supporting 
the instructor, their usefulness is bfased 
on enhancing thrb training process 'with 
regard to cost-effective, considerations. 
Many of these Characteristics will also 
directly impact j the trainee interface as 
well,* since -many of the instructor tasks 
deal with the trainee interface. 

TRAINING J^TfiODOLOGY AIDS ANALYSIS * A 
similar Analysis should bi conducted - to 
determine support capabilities with regard 
to generic .training methodologies (e.g., 
importance of specific external feedback). 
This type of. analysis, which would impact 
instructor and trainee ^interfaces, would 
have a similar series of elements, as 
follows: , A - 

t TraWjig Methodology characteristics 
iwYitification of those specific 
training process characteristics 
that ar£ relevant to the particular 
training situations supported fry the 
training device/tnaining system 
(e.g., immediate graphical 'feedback 
of tactical parameters). This would 
be similar to a task analysis, but 
conducted to identify the specific 
' training methodology characteris- 
tics. The intention is .to identffy 
all those characteristics Of 1 'the 
generic training methodologies that 
may be important for this particular 
training process. C 

t Prioritization of training 

methodology characteristics — each 
of the training methodology 
characteristics should be evaluated 
with , . regard' to Its potential 
effectiveness ■ during the training 
process', for the particular trailing 
device/training system 'under 
, consideration. The result would be 
a prioritization of those character- 
istics on the basis of their likely 
training effectiveness Impact. 

- •» System/device capability analysis — 

the capability of the training' ( 
device/training system to support 
each of the training methodology 



characteristics should N be deter- 
mined. This should be done wtth 
reg^rdto the specific aspects of 
the yu^ining situation and 
methodology characteristics, desired 
to achieve an effective training 
process. For example, if a complex 
tactical problem is being trained, 
del ayed feedback regarding detai 1 ed 
* tactical and performance parameters^ 
may be desired to enable detailed 
analytical investigation of 
parameter interrelationships; the 
abil ity of the training system/- 
training device to provide the 
detailed feedback information in an 
appropriate form (e.g., on a large 
screen graphical display) should be 
evaluated. , * ^ 

t Identify . instructional support 
. capabilities r- based orf the above 
Elements and a cost/effectiveness 
trade-off analysis, specific support 
capabilities would be Identified. 
The results could identify deficient 
capabilities on the existing 
training device/training system, or 
c#uld identify/select capabilities 
to be developed .in a new training 
device. 

The above elements represent an outl ine of 
steps, similar to that of the ISD process, 
for the ' design of the instructional 
subsystem 5 characteristics. Obviously, all 
of the steps have not been detailed, nor 
have their specific procedures been 
adequately developed. Rather, they are 
intended to indicate that a structured 
' process similar to that of ISD should be 
accomplished for each of the major .aspects 
of the instructional support subsystem. 
Furthermore, the instructional support 
subsystem includes several major parts of 
, the training device/training system, each 
of which requires a distinct 'analysis. . It 
is important to note that the above 
analyses should be ^adequately developed, 
but done so quickly with a minimum of 
frills. The-arfalysis should be tailored to 
the available information, <the sophistica- 
tion of ^the training device/training 
system, and the available resources. Many 
of the characteristics could be evaluated 
from the standpoint of a Chopping list of 
4 potentially useful characteristics; and the 
final set selected from the list. 

This approach is appropriate for the .design 
of a new training device/training system, 
as wel 1 as that of improving an existing 
one. SMARTTS is an example > of the latter 
case. The methodologies are likely to be* 
slightly different for each Q^se, but 
basically the same. 
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S MARTTS DESIGN . The SMARTTS instructional 
support subsystem design was the result of 

several analyses similar to the above, over m 
a period of years. The initial concepts of 
SMARTTS stems from an investigation of • 
submarine tactic* training (Hammell, Sroka, 
and Allen, 1971 Hammell , Gasteyer, and 
Pesch, 1973). Of the variety of 
reconmendatixjns ' that were developed 
regarding basic and .advanced submarine 
officer training systems, many addressed 
the need for appropriate instructional 
support capabilities. Figure 2, taken from 
the latter report, lays, out the fundamental 
instructional support capabilities that 
*ere later to be developed under SMARTTV. 
The concepts developed in these- earlier 
investigations' -were later fully developed 
in specific detail on the basis of a front 
end analysis conducted under the . SMARTTS 
project. This later analysis included 
several of the above-recommended steps, 

'focusing on both the instructor tasks and 
generic traioing methodologies. h This 
analysis also selected which of the 
capabilities recommended in the ear net- 
studies were to be developed in the initial 
preprototype SMARTTS, and which will be 
developed in subsequent developmental 
versions of SMARTTS. 

An overview of the instructor function and 
task analysis conducted ..£or SMARTTS is 
presented below. 

Instructor Function and Task Analysis. An 
overview of the analysis ot instructor 
functions -and tasks is presented in Figures 
3 4, 5, and 6. This analysis identified 
the major % functions conducted by the 
submarine tactics instructor, the specific 
tasks he performs with regard to each- 
function, and' estimated' the time loading 
and difficulty associated with performance 
of those tasks. The major instructor 
functions were determined as follows: 

t exercise development 

#. monitor and control of the training 

process * 
t briefing (pre, freeze, post) 
t training system management 

» 

Monitor and control of the training process 
is the instructor function given th£ 
greatest recognition by training device 
builders (Figure 4). When a training 
devic? is employed, the instructor must 
set-up the exercise, .control the device 
during the < exe*cis$ (e.g., maneuvering 
targets), monitor the trainees' activities, 
and provide some amount of training during 
the exercise (e.g., guidance and feedback 
to the trainees)'. Capabilities, in one 
form or another, are provided on most 
training devices to enable set up- and 



control of the exercise. Some capabilities 
are often provided to permit monitoring of 
trainee activities, and often some . 
recording. Occasionally, capabilities are 
available for providing some training 
assistance, such as feedback. However, all 
too often the monitoring «and control 
capabilities installed on a training device 
are designed from an engineering-control 
standpoint, rather than from an instruc,- 
tional process control viewpoint. Specific | 
instructor tasks, A-L, are included in 
Figure 4, along with generic training 
issues and considerations under each task. 
Also, the multiplicity of branching paths 
between tasks -is also indicated to some 
extent by the (A) and (§) symbols. The 
'information contained in Figure 4 is a 
summary of the type of analysis information 
upon which the SMARTTS characteristics were 
based. For example, the instructor has a 
task to monitor the scenario (i.e., Task 
C). A consideration under that task, 
particularly K in a complex system, is 
providing cues ta the. instructor regarding 
current and upcoming scenario events. 
Capabilities were developed in SMARTTS" to 
provide cues and alerts to "the Instructor 
regarding various tactical actions on the 
part of the target, as well as when various 
performance indicators (e.g., probabi H ty 
of ownship counterdetection) went beydnd 
preset standards. These # capabilities, 
which are easily provided by the .computer- 
controlled training device, reduce the 
instructor's scenario monitoring load, 
enabling him to devote a greater proportion 
of time to monitoring " the trainees (i.e., 
Task H). Other examples are given later 
under Training Assistance Technology. 

1 Briefing of ttte trainees (Figure 5), in its 
various forms, is probably the most 
important function performed by the 
instructor. In this function the 
instructor has a direct interface with the 
trainees, and provides them wifeh specific 
information to reinforce and/or modify . 
their behavior. In a simulator-based 
training system briefings may be given 
prior to, during a pause in, and following 
completion of the session on the " simulator 
(i.e., pre, freeze, and .post briefings 
respectively). Although $s§ential to the 
effectiveness of ' the training process, the 
emphasis ' placed on briefings varies 
considerably across training establish- 
ments* Furthermore, the capabilities 
provided as part of the training device/- 
training system to assist in ^nducting 
effective briefings are often quite 
limited. Examples » of capabilities to 
support briefing tasks will be provided 
later below under Training Assistance 
Technology. 



ERIC 



132 



V J ■ 



TRAINING SYSTEM 



TRAINING OUECTlVtS. , , , 



TRAINING OEVICC 



TRAIN7n\J^*OCRAM 



I 

U 

I 

I 

I 

I 

I 

I 

I 

I 

I 

I 

I 



SIMULATION 
SUtSYSTCM 



INSTRUCTIONAL 
SUBSYSTEM 



INSTRUCTOR 



EXERCISE 
DEVELOPMENT 
FACILITIES AND 
CAFAHLlTtES 



MONITOR & 
CONTROL TRAINING 
ON DEVICE 



MANAGEMENT 



BRIEFING 
WE POST. FREEZE 



TRAINING SYSTEM 
' MANAGEMENT 



Figure*!. FocuS of Instruction*! Subsystem 



MEOIACK 

OlSflfY 



HIST0RY7RACK 
DIAGNOSTICS 



FIRE CONTROL 




PERFORMANCE MONITOR 



1 OISERVATIONS 



\^ INDUCTOR ^sj 



PERFORMANCE 
MEASURES 



I . 
CUES' 
SUMMARY 

! 



r — — 






TRAINEE 




PERFORMANCE « 




IIIRARY 



KNOWIEOGEAILE 
' OPPONENT 



Flour* 2. Instructional Support Capability Requirements 
for th^e 2IA Series Submarine Combat System Trainers 
(from Harwell, Gasteyer and Pesch, 1973)* 



ERIC 



133 



TttMM CTt AM M«TT>»V 



tm+hvwc nucnctS data 



(>(*Ott 

oMcnvts 



MllCT 
AmtOTWAT 



• tXTtKUVt AHO 
VAAMO UAAAAT 



• tOt AyAUAftK 

• ACClSlAAturv 

TO TACTtCAl. 

• ttHAVtOAAk 
STAinOAROI 



0€*CM 
CI **f *AL 
tCIIAAlO 



•mi AH TO-5 

WIH fV(MT{ 
JITUATTOW 



1MOK 
ICtHAA'O 



• TAACJTS 

• M ATOMS « 



• CMVlAONMCMT 



CMAMUS 
«< MUCTIO 

tttacrUt 



• (AM O* MAIUfeC 
CMAWC4S 

• CA#A*L»TV TO 
MCOAO 

• nM itrvC* vf o 



• »u*t.»c»T» o* 



• S£H«*T1*C 
TAMCin 

• SCWtt*c Of 



• O^IMAV o# 

UAT( JUAL 
. » TO 1 ? 



• CAlCWATt 
"1 AA)0 

TACTtCAl VA*IA*4(l 



• l»»vC*n&ATf 
A(.T(«NAT<Vti 
*( TftAI*<*G 
VAIKA aa-o 
*Avit* 



• HtOTtCTK* 
ACAtttST tOO O* 
(StSTANT 



I. 



M cure 3. Exercise Develorxnem 




—00— 



• <OM»MT( 
1 • i 



Hour* 4. ^Monitor ind Control Tr*1n1no 



ERIC 



t "J. 



MIWi 
«NTAT> 
• MMTAUCTOM 



AVAHAM.ITY 
• MOTMJCTOA 



OCT AMI Mflifl 



A 



0- 



1 



OCTd 
TAA 
MM 



• TACTKA1 



• (AH •# 

MTTH*C 



• OSJtCTtvt 

• MCjtcnvf 

• I AH 



IVAUMTt 
TAAIMf 

unto 



• *AGMO«T< 



• HMOMIAWCt lilCAlO 
■Mtmi MUMCTTVt) 



Floure 5. 8rie'in<j (Pre, Freeze. Post) 



0- 



• CTO*f 

WITHOUT 
A/f«CTM« 

omunNQ 



iowo~rt am 

MUAftCH 
CA/AMUTY 



MTtMMMt 

uorimcATtvt 
OMCTTVIS 



• |AU AM* 

'UJDWUTV 

• TUN 



Ovmrr 

DATA 



MTVftTMATI 
fVAM 
|X»f«MHIfTM 
Of WO* 



• IMO*(tOUAl 

• CM* 



COMOOCT 
STATTITICAt 

nsn 



|VAIUAT( 
AftAlVM 
•ATA 



• C*A»MC 

• T A*US 



• tUtT-M 
ITATTtTKAl 
ftOVTMtfS 



0- 



• HUT HVUTt ' 

• »UIT MtMACX 



• fcltot fOA 
A*0*n<MAl 
TRAMMttC 



WCQA— I*»AWO« 



• TftAJMMC 

*f MOMAAftCf 
MMOftMATION 

• *tt(T N*M4 



TfUUWMG^ 

mrtw 

COWTROl 



• MVS to». rirr 
amo •*rrM»trrc 

MATlftUl 

• MTTHUCTOA 



• mmmoctki 

• T RAJ Ml MO ttilCV 



• TAAHWMO 
tVSTIM 
MMAACH 



• ISTAA4KH 
THAI Ml MO 
4 STAMOAMMt 



! Flovref . Tnfnfno System Pjwoefnent 



135 



ERIC 



1°7 



. .. 



Exercise development (Figure 3), which 
proceeds the conduct of training sessions, 
consumes a substantial portion of an 
instructor's time. This is a complex 
function wherein the instructor actually 
develops the training program and its 
supporting materials. It is complex in 
# that it often requires the creative design 
of the course atrd its training exercises, 
with the instructor drawing upon the 
state-of-the-art in training methodology to 
achieve an effective training process. 
Whereas some capabilities, although usually 
quite limited, are often provided on the 
training device/training system to support 
exercise monitoring, control and briefing, 
usually little if any capabilities are 
^ provided to support exercise development. 
It is the exercise development function, 
interestingly, that determines how the 
training device will be employed within the 
training system. Examples of system 
, capabilities to support exercise develop- 
4 ment ace presented 1-ater under Training 
Assistance Technology. 

Training system management, similar to 
exercise development, is typically an 
overlooked function of the instructor and 
one with' which he spends substantial time. 
A wide variety of tasks, issues, and 
considerations may be associated with 
training ~ system management. Furthermore, 
each of these ^may depend on the particular 
training establishment under which the - 
trai ni n^-devrce/trai n i ng sys tern operates . 
Nevertheless, several training system 
management tasks are common 'across training _ 
establishments, as indicated in Figure 6. 
Perhaps the most important common set of 
tasks involves monitoring the effectiveness 
of training over time. For example, it is 
highly desirable to identify those training 
objectives (i.e., with associated 
exercise's) that the typical trainees can 
readily perform* upon entering the training 
program; and likewise to .identify those 
training objectives (and exercises) that 
- the trainees typically* have substantial 
difficulty «with y even near the end v of the 
training program. Ideally, reduced 
emphasis would be placed on the former 
while increased emphasis would be placed on 
the latter training objectives in subse- 
quent training programs. For another 
example, alternative training, methods and 
training materials should be periodically 
evaluated to continually upgrade their 
quality and the effectiveness of the 
training process. These are major 
management tasks directjy impacting the 
cost/effectiveness of trainings and 
subsequently the operational readiness and 
effectiveness of. weapon systems. 

\,Each time a training exercise is run on the ' 
simulator valuable training performance 



data is generated. These data could be 
collected over time and used as the basis 
for the above evaluations, and other 
training system development activities. 
These data would be extremely useful to 
most levels of management for evaluation 
and planning purposes. Relatively lu ^ le 
of these data are recorded and used today 
with regard to most training device/- 
training systems. This is one example of a 
set of training device/training system 
capabilities • that could substantially 
augment training system management. 

The instructor function and task -analysis 
conducted on SMARTTS was integrated with 
the generic training methodology analysis 
as indicated in Figures 3 through 6. This 
analysis resulted in the identification of 
issues and constraints deemed important to 
the conduct of an effective training 
process for submarine officer tactics 
training. These form the basis for the 
subsequent design of the SMARTTS preproto- 
type system (i.e., the instructional 
support subsystem capabilities which make 
up SMARTTS). A similar process should be 
conducted as part of the design for each 
training device/training system. 

The next step of the design process is the 
trade-off analysis to identify and design 
the specific capabilities to be included in 
the particular instructional support 
subsystem for the particular training 
device/training system. This trade-off 
analysis should be conducted similar to 
that of other cost/effective analyses done 
under the ISO' process. The results of the 
analysis conducted for SMARTTS, although 
not the analysis itself, will be summarized 
later in this paper under Training 
Assistance Technology. 



Other Design Issues . The above recommended 
analysis ' has as its objective 'the 
development of the characteristics of the 
instructional subsystem from the stand- 
points of the instructor functions and 
application/enhancement of effective 
training methodologies. Several other 
issues should also be considered during the 

- * design! of * th'is subsystem. Of primary 
importance is the flexibility of the 
instructional subsystem with regard to 
future upgrading. That is, the subsystem 
should be designed with evolution in mind. 
As the training device/training system is 
used, and as information is generated 
regarding the effectiveness of its various 

! • charcteristics, the instructional subsystem 
Should be continually improved. More 
effective training methodologies should be 
incorporated, new performance indicators 
, and feedback displays developed; and so 
onJ . As /the instructors 1 continually 
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Increase their experience In -training and 
with the device/system, as the operational 
systems continually change, and as the 
trainees continually Improve their 
performance the training device/training 
system should likewise change, to ref ocus 
training emphasis and to continually 
Improve the effectiveness of training. The 
flexibility, to permit evolution in this 
regard must be designed-in at the onset, 
with a conscious effort made to continually 
upgrade training. The'" engineering 
characteristics of the system should be 
such that the ** major training-related 
characteristics can be readily modified, 
and new characteristics adde'd. This 
impacts the design of the hardware 
components and the software architecture. 
For example, new performance indicators and 
feedback displays will always be required. 
Hence, the software architecture should be 
• designed to permit easi ly adding new 
performance indicator algorithms, which may 
require access to a wide variety of 
scenario parameters; generic display 
formats should be set up which enable the 
construction of new displays drawing upon 
the variety of parameters generated and 
recorded, and a variety of graphical 
formats. Although details of system design 
flexibility are not discussed in this 
paper, they have teen incorporated into the 
SMARTTS system. It is anticipated that as 
SMARTTS is used demand for new capabilities 
will - continually exist. The SMARTTS 
software architecture is designed to 
readily accommodate such modifications and 
additions as relatively minor software 
changes. Some of the capabilities 
identified under the SMARTTS front-end 
analysis" that would further enhance this 
evolution/modification process have not 
'been included in the preprototype, but. are' 
obviously considered* for the " further 
developmental units. 

It is apparent, as indicated above, that 
the training process conducted using a 
particular training device/training system 
will change over time. Furthermore, in a 
similar vein, at any given point in time 
the complex simulator-based training system 
is likely to be used for a wide range of 

-—training levels and training needs. Hence, 
system flexibility should also address the 
ready tailoring of the instructional 
, support capabilities to a particular 
training situation. That is, different 
types of support capability characteristics 
are required for the different levels of 
training likely to be encountered. Many of 
these can be determined in advanced, while * ' 
others must be tailored close to the time 

■ of initiating the training exercise. The* 
training device/training system, and its 
instructional , support . subsystem a should be' 
flexibly designed to .permit on ''the' spot 
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tailon^ of the training exercise, 
performance indicators used, feedback 
displays and their .formats, and so on. For 
example, it may be necessary to generatrT^N. 
modify the next training exercise \ 
specifically on the basis of performance fh 
the previous training exercise. JUso, the 
performance standard on which a cue is 
presented -to the instructor may likewise 
hrave m to be modified shortly prior to 
running the exercise. - The system 
architecture should be flexible to enable 
this type of tailoring with relatively 
little effort. 

As noted in the introductory paragraph to 
this paper, human factoring of the 
instructor and trainee interfaces is as 
important in the training device/training 
system as it is in the operational weapons 
system. Too often this aspect of design is 
overlooked. Many of the instructional 
■support capabilities provided to the 
instructor could simply be included under 
the category, of "good human factors 
design", since their main purpose is to 
facil itate the. instructor/system interface 
and improve the effectiveness of his 
performance. This includes, for example, 
the specific human computer/training device 
interface for exercise set up, control and 
monitoring. An effort "should be made to 
minimize the amount of training 'necessary 
m for the instructor to be able to operate' 
the training device. On SMARTTS, for 
example, the traditional alphanumeric/- 
function-key irtput devicp* was replaced with 
a touch-sensitive/ plasma display dev4ee. 
The plasma input devi.cg has many advantages 
beyond those 'which have been incorporated 
into the preprototype SMARTTS; neverthe- 
less, those plasma-related input features 
incorporated into SMARTTS 'should provide a 
substantial increase in the effectiveness 
with which the instructor can communicate 
with the training device. The instructor 
does not have to learn, for example, a 
large number of input codes for effecting 
control of the training device; he does not 
have to remember the options available to 
him, and look up the appropriate codes, or 
coordinate the options from the CRT to the 
alphanumeric ■ input keyboard; he does not 
have to input a sequence of alphanumeric 
commands on the keyboard. Rather, the 
complete set of input commands ' are 
logically layed out in a tree structure, 
and only those commands available tfr him at 
any point in time are displayed on the 
plasma entry device. These available 
cofomandS are displayed in English. To 
enter a command, he simply points with his 
finger to ,the appropriate command. This 
type of interface is user friendly, and to 
a large extent self- teaching through use of 
the device itself. Careful consideration * 
should' be given to tj^ many other aspects 
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of the instructor and trainee interfaces 
with the training device (e.g., location of 
instructor information, location of trainee 
feedback displays). 



Training Assistance Technology 

Jhe remainder of this paper identifies' 
aspects of training assistance technology 
that *pe important elements of the 
instructional support, subsystem. These are 
presented with regard to each of the four 
major instructor functions identified 
above. Specific examples of characteris- 
tics are given, pertaining to the SMARTTS 
system, where applicable. 

EXERCISE DEVELOPMENT CAPABILITIES . (Figure 
jyi Exercise development may include the 
development of the training objectives, the 
course syllabus, and so on. Although these 
capabilities can be supported by parts of 
the instructional subsystem, under this 
function and under the training system 
management function, they are not directly 
addressed - herein. Rather, those 
capabilities more directly related to the 
training -device activities itself (i.e., 
conduct of scenario exercises) are 
specifically addressed <~ herein, and 
summarized in Figure 3. . 



Instructional Support Material . A variety 
of ma terial should 5e available to the 
instructor as his primary resource for 
developing the training device exercise 
scenario. Most of this material would 
likely be in the form of Handbooks, 
although some .could be available in a 
computer data base. This information 
should include (1) a complete set of 
tactical and behavioral . training 

j)6jectil»?s, cross-referenced to exercises 
and operational situations; (2) "a complete 
set of tactical reference information, such 

*as Naval Weapons Publication (NWP) series 
documents; (3) a set of guidelines 
regarding training methodologies, training 
device operations, and training exercise 
development; and (4) trainee input 
characteristics information, including 
previous trainee/team performance 
information, and training/tactical needs. 
The Information related to trainee input 
characteristics could £e available on a 
large information base assembled from 
previous training exercises of that 
particular individual or team, together 
with comparative data on the population of 
similar trainees, and so on. The necessary 
data would "be collected under the training 
system ' management function of the 
instructor. 



The ~ SMARTTS system, to support this 
exercise development function, has 
developed documents to aid the instructor 
in the design of exercises. SMARTTS has 
developed not only the standard system 
operating handbook, but also an 
instructor's handbook which addresses how 
to design exercises, alternative training 
methods that can be employed, how to 
conduct a training exercise, and so on! 
Additionally, a comprehensive set of 
training objectives have been developed 
along with a set of training exercises 
cross-referenced to these objectives. A 
training structure has been developed 
across multiple . levels of trainee 
proficiency, with diagnostic information . 
supplied to evaluate the trainee/team input 
characteristics; remedial training 
activities have also been included to help 
assure that all trainees meet the mininum 
entry level- standards. Furthermore, . 
training materials for the initial tactical 
course, including exercises, have been 
developed utilizing the SMARTTS capabil- 
ities. This approach not only has provided 
the structure and guidelines with which 
instructors can develop subsequent training 
courses and exercises, but has also 
provided an example in the form of the 
iqitial such course. 

Exercise Development Simulation . A 
flexible high speed simulation capability 
is nece'ssary to enable the instructor to 
rev -j ew existing exercises in the library, 
modify an exercise, or create a new 
exercise. This fast time simulation 
capability is necessary in support of tasks 
"B" through "I" under the exercise 
development function (Figure 3). Review, 
modification, or development of an exercise 
can be an extremely cumbersome and time 
consuming process. This is particularly 
the case where calculation of the 
developing t4fe line scenario interaction 
is necessary^fco* assure appropriate scenario 
events in support of the Exercise 
objectives. Furthermore, it is well 
recognized that scenarios often turn out 
substantially different when run on the 
device than as they appeared on the desk 
top plot sheet of the developer. The fast* 
time simulation should provide capability 
for the instructor to: 

(1) call up an existing exercise and run 
it in fast time to any desired 
point, retrace steps, and so on; 

(2) investigate alternative actions from 
any desired time to any other 

* desired time; 

• (3) generate and display relevant 
performance indicators and situation 
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parameters (e.g., tartet range) at 
the various times 01^ the actual 
scenario and the alternatives under 
investigation; 



(4) modify any of the scenario • 
parameters, enabling comparison of 
the tactical parameters « and 
performance indicators at any 
-subsequent point in time; 

(5) enter new performance indicator 
algorithms to be recorded and 
displayed during or subsequent to 
the exercise; 

(6) insert instructor cues to be 
automatically keyed at times during 

m the exercise based on various 

aspects of the scenario parameters 
and/or time; 

(7) configure new feedback display 
formats for trainee briefings based 
on the training objectives, issues 

. to be focused on, or parameters 
generated during the scenario, and 
so on; 

(8) develop new subjective performance 
monitoring input categories for 
observation entry by the instructor 
during the actual exercise; 

(9) set up all necessary scenario 
parameters for running the scenario 
on the training device; 

(10) permanently record the exercise in 
the exercise library for later 
review' via this fast time simulation 
capability, or for Actual running 
during a training exercise on the 
training device. 

Additional capabilities couTd -also be , 
listed; but these present a good overview 
of the type of capabilities that should be 
available to the instructor in an off-line 
mode of the training device to enable him 
to review** and configure scenarios. A 
library of all available training exercise 
scenarios should obviously be accessible to 
the instructor for this development 
process. This fast - time simulation- 
capability should be available 1 at an 
off-line location to the training device, 
permitting access and fast time simulation 
in parallel with the actual running of the 
training device. The terminal should have 
appropriate graphical displays and an entry 
device. to facilitate instructor interaction . 
and^evaluation. 

SMARTTS has identified the need for a fast 
time simulation capability to assist the 
instructor . in developing exercises., This 



full capability would require the fast-time 
simulation capability as well as other 
elements of the exercise < development 
function. In the preprototype SMARTTS this 
„ capability was given secondary importance 
to the monitoring, control, and briefing 
capabilities. Nevertheless, a fast- time 7 
simulation capability at a remote terminal 
was provided on SMARTTS, partly due to a 
particular design of the system which has 
two parallel complete operating systems 
with associated simulation models and two 
complete instructor consoles (i.e., one in 
a classroom and the other in an attack 
center). The SMARTTS system permits the 
fast- time running (i.e., at a rate up to 
16X) of any exercise scenario in the 
library. All of the tactically relevant 
variables and performance indicators can be 
evaluated on either of the instructor 
consoles. Modifications of any of the 
tactical parameters can be effected from 
these consoles, with the modified exercises 
permanently stored for later training use. 
This SMARTTS capability, during the limited 
time in which it has been given operational 
use, has been found to greatly assist the 
instructors in preparing and evaluating 
scenario exercises. Requirements had been 
earlier identified to more fully develop 
this capability in the later developmental 
versions of SMARTTS; initial indications to 
date are that these planned additions 
should be carried out. 



Instructor Interface Language . The typical 
instructor in military training systems has 
a good operational background (e.g., 
submarine operating experience for a * 
submarine tactical instructor), although a 
limited instructional and computer 
background, particularly with regard to 
operating particular , training devices. 
Hence, it is desired to have a, computer 
language developed to facilitate the 
instructor's interface with the training 
device. This interface lanuage should 
permit the instructor to interact using 
somewhat standardized operational 

terminology, instructional terminology, 
and/or near-English terminology. It should 
be designed to accommodate those activities 
normally performed by the instructor when 
interfacing to the training dev.ice. With 
regard to exercise development, for 
example, this -language should accept* 
discriptive commands by the instructor in 
the form of the normal operational 
parameters (e.g t , whereas the computer 
manipulates targets on an X, Y, Z grid,, 
operational personnel often " view the 
geographic situation in terms, of range", 1 
bearing, ,and depth/Qva.luation). Further- 6 
more, the - interface language should be 
structured to contain a large set of mac**o , 
function* that readily translate the 

y 
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instructor's needs into exercise, 
performance indicator, cue, feedback 
display, etc., modifications. For example, 
the capability should be provided to enable 
the instructor to readily tailor feedback 
displays to particular exercises. This 
macro function should enable the instructor 
to rapidly provide the information 
necessary to configure the display in terms 
as close as possible to thos* operationally 
used, *nd then enable the computer to 
automatically configure the display. This 
might be accomplished using a digital 
interface drawing tablet, with appropriate 
parameter data readily accessible, and with 
standard ized mani pul at ion al gon thms 
available. ■ 

SMARTTS , although not oossessing 
capabilities to this extent, does permit 
instructor flexibility in setting-up and 
tailoring trainee information displays. 
SMARTTS, for example, provides several 
generic plot formats of X versus Y with 
which the instructor can display any 
parameter (i.e., performance indicators and 
tactical variables) as a function of any 
other parameter. The instructor simply has 
to indicate which variables he wishes to 
plot and they would be automatically 
plotted. The instructor can also set-up- 
several groups of > parameters to be 
called-up for display simultaneously by 
requesting the group(s) rather than each 
individual parameter. These capabilities 
are not only useful during the exercise 
generation function, but are also intended 
for the briefing function. SMARTTS has 
identified the need for an instructor 
interface language,. for exercise 
development, bat has not formally developed 
stfch'a language as part of the preprototype 
system, although some capabilities of this 
type are included in the preprototype. 

MO NITOR ANP* CONTROL TRAINING . The 
instructional support subsystem should 
provide a predominence of its capfabvlites 
for the monitoring and control of training, 
and for the briefing function which ' is 
addressed later. Suggested monitoring and 
control capabilities are presented below, 
in reference to Figure 4. 



Exercise set up . " At the point when the 
scenario exercise will* be run on the 
training device during actual training, it 
^shouj® have undergone prior development and 
evaluation. Hence,' typically, 'exercise set 
up would be a matter of simply selecting 
the appropriate exercise from the library 
*nd initiating the problem on the training 
device. Some minor modification may also 
vSe desired at * this time., An exercise 
library, therefore, should be available to 



the instructor from which he can* select the 
appropriate exercise. Actual set up on the 
device should be as automated as possible, 
so as to free the instructor to -perform 
other duties, and to enable quick turn 
.around time between training exercises if 
desired. The exercise set-up procedure on 
today's training devices, even after the 
exercises have been fully developed, is 
often cumbersome and time consuming; manual 
entry of all the set up^ parameters is often 
necessary at the time of initiating the , 
exercise (e.g., in some instances this may 
take up to 45 minutes). This laborious 
entry process should be unnecessary for 
standard exercises; on a sophisticated 
training device the exercise parameters ,can . 
simply be stored'on a disk for automatic 
entry. A capability should be provided to 
enable the instructor to modify any of the 
parameters at the time of initiation-, if he 
so desires. 

SMARTTS provides an instructor's console 
iwnediately adjacent to the '/ire control 
system consoles from which the instructor 
can select an exercise from the exercise 
library and have it automatically entered 
upon conmand. He can also investigate and 
manipulate many of the relevant tactical 
variables at this \ time to • modify the 
exercise. Furthermore, if the instructor, 
so desires, a modified copy of the exercise 
can be automatically stored in the exercise 
-^library as a new exercise for later use. 
This type of capabil ity substantially 
reduces the low level time consuming task 
of set-up. 



Exercise Monitor . The instructor must 
monitor both the scenario and the trainee 
activities. Relevant monitoring informa- 
tion should be generated by the training 
device, "with, much 4 of t,he information 
selectable by the instructor in real-time 
when he requires it. It should be provided 
to him in a timely manner and in a clearly 
understood and meaningful format. 'Cues 
should be provided as appropriate to 
off-load an appropriate part of his 
monitoring function. This information 
should be provided to the instructor in a 
convenient location which-^ enables him to 
perform his other duties with a minimum of 

* travel. Both alphanumeric and graphical 
information should be provided, depending 
on the purpose of the information and the* 
precision the instructor requires. 

* SMARTTS provides an , instructor 1 s console in 
the attack center, immediately adjacent to, 
the fire control party and the MK117 fire 
control system. An identical instructors 
console » is located in the classroom for 
briefing purposes. SMARTTS . automatically 
records a wide variety of tactical 
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variables and performance indicators. This 
information is available to "the instructor 
upon request at the instructor's console. 
Both graphical and alphanumeric formats are 
'provided, depending upon the Information 
and the instructor's desires. The display 
is automatically up dated over time to 
provide the most recent information, as 
well as to provide historical information 
from the beginning of the exercise or other 
designated times. Cues are. provided to the 
instructor" regarding (1) performance 
Indicator values or tactical parameter 
values that have gone beyond predetermined 
limits (e.g., probability of counterdetec- 
tion going beyond 50 percent); and (2) 
pending action by the automatic interactive 
target (AIT) (see below regarding control 
capabilities). The display formats used to 
provide information to the instructor were 
developed with regard to the requirements 
of the training situation and the type of 
information' submarine officers normally 
deal with (e.g., line-of-sight diagram). 
Hence, the instructor need not return to 
the program operator consoles to monitor 
exercise progress. Rather, he can stay 
with the fire control party to monitor the 
trainees 1 actions, and still have the 
normal scenario monitoring information 
available. Furthermore, SMARTTS provides 
the instructor with a variety of additional 
information concerning trainee performance 
and the problem status. 



Data Recording . The analysis of instructor 
functions and tasks -revealed that typical 
instructors spend a relatively large amount 
of time recording exercise data, if they 
provide this type of' feedback to the 
trainees after the exercise. Furthermore, 
the better postscenario briefing sessions 
are those in which the instructor has 
substantial information available for 
presentation and discussion with the 
trainees regarding various aspects of the 
problem. Much of the information recorded 
by the instructor is normally generated by 
the training device (e.g., tactical 
parameters and events). The typical 
computer-based training device has the 
capability to record these data for later 
accessing by the instructor during the 
postscenario briefing. In addition, a 
considerable part of the instructor's tasks 
is to monitor and record actions of the 
trainee- that could not be automatically 
recorded by the system (e.g., communication 
between team members, coordination). These 
types of subjective performance indicators 
would be relatively difficult to achieve 
via automated means in many training 
situations. Hence; the . instructor should 
focus a substantial portion of his time 
monitoring and recording, these types* ,of 
performance indicators, for later feedback. 



The training device could assist the 
instructor by facilitating his recording of 
this information via a standardized 
interface with the training device. 

The SMARTTS system records a wide variety 
of tactical variables that occur during the 
exercise scenario. Additionally, a set of 
objective performance indicators are 
generated and also automatically recorded 
during the exercise scenario. Finally, a 
capability is provided for the instructor 
to enter a variety of subjective 
observations into the training device/- 
computer storage facility. For example, if 
the instructor observes highly proficient 
communication between the fire control 
coordinatbr and the plot coordinator during 
a particular time of the scenario regarding 
a particular aspect of the scenario, he can 
easily enter this observation into the 
computer by designating the appropriate 
observation. All of the above" data are 
readily accessible to the instructor during 
the exercise, and also following the 
exercise for discussion in the briefing 
session. This set of capabilities not only 
reduces the instructor's load, but also 
provides , tools' to the instructor for 
training that he here-to-fore did not 
have. For example, the instructor 
typically was unable to monitor changes in 
the fire control system solution accuracy 
over the course of the exercise as a 
function of ownship's maneuvering; this 
information is now automatically recorded 
and can , be accessed by the instructor at 
any time, including during the postscenario 
briefing session. 



Scenario/Exercise Control . The instructor 
should Fe provided with a witfe range of 
control capabilities to enable him to 
readi ly control all aspects of the 
scenario, and also control presentation of 
pertinent information to the trainees as 
necessary. The control capabil ite* should 
be provided in a convenient location, 
similar to that for the monitoring 
information, preferably near the trainees 
such that the instructor can continually 
monitor the trainees and the problem while 
entering necessary - control commands. 
Ideally, the instructor would spend a 
minimum of time controlling the exercise, 
devoting most of his resources to other 
training-related activities. In this 
regard, scripted scenarios should be 
available wherein the target has predeter-* 
mine actions. Additionally, other types of 
Control for the target and other aspects of 
the problem would be desirable if they 
freed the instructor from these tasks. It 
should be * noted, however, that the 
instructor sljpuld be allowed to control, any 
of the aspects he so desires. In^addition 
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to the scenario control capabilities, the 
Information presentation capabilities 
should likewise be located for appropriate 
viewing by the trainees, and so that -the 
instructor can control the information 
presentation via convenient snd effective 
media. The location of this information 
presentation ' (e.g., feedback) would 
notaally be, either in the operational 
simulator 'environment (erg., attack center) 
and/or in a briefing room. In either case, 
the instructor should have complete control 
,over the information presented, enabling 
*'him , rapid configuration of the desired 
displays. 

The instructor's console in the SMARTTS 
system is located near the fire control 
party in the attack center. The instructor 
is presented with monitoring information on 
a color "graphic CRT as well as the plasma 
display entry device. The instructor can 
control many of the relevant tactical 
problem variables (e.g., Maneuver targets) 
from this console, as werl as control a 
wide variety of information to be presented ^ 
to the trainees. The information 
presentation to the trainees is achieved in 
the attack center via two overhead-mounted 
color CRT displays. A variety of display 
formats can be put up independently on each 
of these CRTs via command from the 
instructor's console. The similar 
instructor's console located in the 
adjacent classroom controls the information^ 
to be displayed on a large screen display 
during briefing sessions. 

SMARTTS has the normal scripted scenario 
capability, along with, an automatic 
interactive target (Alt). The AIT 
capability provides a computer-controlled 
model of an erremy target platform. t-This 
model is based on the best available 
intelligence information for the particular 
platform. The model automatically controls 
the target in response to s the evolving 
situation events in real-time. Cues are 
provided to the instructor prior to any 
action on the part of the AIT, enabling the 
instructor to (1) allow the AIT to carry 
'out its planned action, (2) override the 
AIT's planned action by having it continue 
doing it current activity, (3) enter a 
different target course of action, or (4) 
take over manual control ^of the target. 
The AIT capability is intended to reduce 
the instructor's load in controlling the 
principle - target, which has been observed 
to be considerable. The AIT, furthermore, 
briiigs in the best available intelligence 
information to probabilistically control the 
target in* a realistic fashion. This 
capability will assist in off-setting 
experience differences and biases between 
instructors.' 



* Information Pcpsentation * Information 
presentation to the trainees, in the form 
of a preproblem .briefing or postproblem 
feedback, - is~ an' -essential part of an 
effective training process. Appropriate 
information must be generated by the 
training system, and presented to the 
trainees in an effective way to assist 
their assimilation. A wide variety of 
media is available for presenting 
Information to the trainees. The most 
common is that of verbal presentation by 
the ; instructor. Unfortunately, 'verbal, 
' presentation is limited with regard to 
handling complex relationships. The use of 
augmenting graphical information (i.e., 
pictures) and examples has been shown to 
substantially increase the effectiveness of 
training (Lesgold, Pellegrino, Fokkema, and 
Glaser, 1978). Hence, a visual information 
display capability should be made available 
in the instructional subsystem, particu- 
larly for complex training situations. A 
variety of capabilities may be associated 
with that information display, such as a 
fast time model to explore alternative 
actions in a .given problem.. Also, a 
. variety of information should be available 
in the training device regarding the 
recently completed exercise scenario for 
presentation to the trainees. Ready access 
to, and flexible control over, this 
information and these capabilities should 
be provided to the instructor at an 
appropriate location. Obviously, the 
monitor, record, and control capabilities 
act together with the information display 
capabilities to provide appropriate 
information .to the trainees. * + The 
information presentation capabilities are 
further discussed below under the briefing^ 
function which overlaps considerably with 
'this monitor and control function. 

The monitor and control capabilities of the 
instructional subsystem are quite extensive 
on SMARTTS, as part of the submarine combat 
system trainer. These capabilities are 
generic, although their specific character- 
istics have been tailored to,, the submarine 
tactics training problem./ The above 
discussion presents a very general summary 
overview of the particular characteristics 
incorporated^ in SMARTTS for this instructor 
function, and its associated tasks, issues 
and considerations. 



BRIEFING . The briefing of trainees is a 
major training process function of the 
instructor. It obviously overlaps with the 
monitor and control tasks as noted above, 
and also overlaps with the trainee 
interface requirements of the instructional 
subsystem. It is the opinion of this 
author that many of the trainee interface 



characteristics of the Instructional 
subsystem are the same as those required 
for the briefing function. The briefing 
function' of the instructor deals with 
providing external information to the 
trainees to correlate with their actual or 
desired actions. This information is a 
primary means by which training actually 
occurs. t For example, this information 
provides feedback informing the trainee of 
how successful his particular actions were 
with regard to the operational objectives. 
Since a major objective of training is to 
develop an awareness on the. part of the. 
trainee of the relationship between 
available information, alternative actions \ 
that can be taken, and t/ie resultant likely 
situation outcomes, external information 
presented to the trainee is extremely 
important. Hence, the effectiveness with 
which information can be presented to and 
assimulated by the trainee will have a 
direct bearing on the Effectiveness of the 
training process. Information presented to 
the ' trainee attempts to achieve the 
following: 

(1) explain a particular evolving 
situation (e.g., events that occur 
during a tactical encoqnter), 

(2) identify . possible alternative 
situations and actions (e.g., the 
types of targets likely to be 
encountered in this area, each 
.target's likely. set of tactics, and 
appropriate defenses), 

(3) the outcome of .the trainee's 
particular actions during' the 
problem (e.g. , the trainee teams 
tactical approach on a target), 

- (4) the relationships between tactical 
variables that were relevant in the 
problem '{e.g., the relationship 
between ownship speed and 
probability of counterdetection), and 

-(5) the likely impact of various 
alternative trainee actions on the 
situation outcome (e.g., the impact 
of different ownship speeds on/ 
achieving a successful target 
approach). These categories of 
information might be differentially 
brought up -prior to a real-time 
training exercise, during $j> freeze 
pause in that exercise, or following 
the exercise. 

The, information categories rail into two 
general classes. The first is to provide 
information regarding a ' particular 
problem. This is typically in the form of 
feedback after the problem occurred. It 
may involve, dissecting the problem to 



investigate the trainee's actions, why he 
took those actions, what options were 
available to him, ahd sro" on. The : second 
class of information is somewhat 
independent of the particular actions on 
the part of the trainee. Rather, it deals 
with the general set of possible problems, 
investigating relevant issues surrounding a 
particular set of -problems. This would 
normally occur in a ^briefing session prior 
to the real-time exercise on the device. 
It may also evolve during the postproblem 
briefing session. Whereas the former class 
of information presentation class relies on 
the data generated and -collected during the 
training exercise, this latter relies on a 
fast time modeling capability to generate 
the various alternative actions, tactical 
parameters, and performance indicators. 
This briefing capability should be located 
where most convenient to the trainees, and 
conducive to the types of activities 
occurring during the. training process. 

Information displays in the SMARTTS system* 
have been provided both in, the attack 
center and in the classroom. These two 
locations recognize the different 
activities that may take place in both. 
The classroom can operate in conjunction 
with the attack center (e.g., monitoring 
the ongoing exercise in the attack center), 
or both can ^ op'erate independently on 
separate problems. A variety of 
performance indicators are available for 
display to provide the trainees with 
information concerning the various aspects 
of their performance. The intention - in 
SMARTTS has been not to develop performahce 
indicators pertaining to good or bad. 
performance, but rather to generate 
information that can be meaningful from the 
standpoint of informing the trainee about 
the scenario and his performance. Often, 
performance indicators , change in an 
opposing fashion, necessitating trade-offs 
between them when selecting the appropriate 
tactical action. From a -training process 
standpoint, the important consideration is 
not how well .the trainee did, but rather 
that the trainee understand the impact of 
his actions and the other actions available 
to him. SMARTTS provides the instructor 
with a wide range of capabilities to access 
the various information that was recorded 
during the training problem. «A variety of 
information 'display, formats are available 
for presenting a * and discussing this 
information, as " well as a range of 
flexibility to "dissect the information and 
focus on particular aspects of the 
problem. The instructor's subjective 
observations, which were entered during the 
exercise, are also available for a 
reference during the post problem briefing 
session. Also, time-flag -units are 
available at several trainee locations to 



opable^e trainees to place a time-tagged 
indication in the scenario recorcjing at a, 
paint when they were concerned with a 
particular issue and unable to bring it to 
the intention of the' instructor. These 
trainee-instituted time-tags can be 
accessed during, the postproblem briefing 
session for investigation, 

Th^ • fast-time simulation, capability, 
discussed above several times, is also 
available in SMARTTS to generate problems 
for presentation and discussion. The 
fast-time capability may be used in a 
preexercise briefing wherein the trainees 
can be given a preview of the particular 
exercise^ they will encounter, pr other 
relevant' exercises. Various performance^ 
indicators and tactical parameters can be 1 
investigated at this prebrlefing time to 
focu$ on the aspects of importance during 
the problem. Alternative sets of ownship 
and/or target actions can also be addressed 
at this time to explore their impact on the 
various tactical parameters and performance 
indicators. This capability is also 
available during a problem freeze and 
during *the postproblem briefing session. 
During the latter session, alternative sets 
of 'actions may be explored for the 
particular problem just completed in ,the 
attack center. It is important to note 
that models are provided in SMARTTS to 
generate information that would ndrmally be 
generated by the trainee team (e,g,, target 
motion analysis solution). This informa- 
tion, generated in' fast .time, is useful for 
comparison w^ith the actual values" generated 
by the trainee team with regard to .the 
alternative actions being investigated for 
comparative purposes. 

It is important that an adequate human 
factors design be implemented fora the 
instructor/computer interface with regard 
to the briefing capabilities. At this 
time, the instructor is standing on-stage 
and has to perform in a. timely fashion. 
These briefing capabilites provide the 
instructor with a substantial amount of 
information to access and present v to the 
trainees. However* it must be facilitated 
irvvsuch a way that it can be brought up 
flexibly and quickly, with relative ease on- 
the part of the instructor. It must also 
£e presented clearly to the trainees. The 
design of the SMARTTS instructors console 
(e,g,«, a color graphic CRT and a plasma 
display entry device), as noted above, has 
been designed to facilitate this interface/ 



Training System Management 

A variety of capabilities are desirable for 

training system managment. The most 

important capability that the device 



automation can assist is the recording and 
storage of data generated during exercise 
scenario runs and the statistical analysis 
routines to enable investigation* of these 
data at a later time. As noted earlier, 
important training system-related data are 
generated each time the traioing device "is 
used. These data can be extremely useful 
in evaluating • and upgrading a variety 6f 
aspects of the training process (e.g., 
development of exercises as noted above )i 
The SMARTTS front-end analysis has 
identified the need for' many of thesV 
capabilities. The preprototype SMARTTS 
System has the capabi 1 ity to record and 
long-term store exercise data for late^ 
analysis. Additional analysis capabilities 
should become a part of the- later SMARTTS 
develbpmental units, 

SUMMARY 

The training device is more than a 
simulator. It has both simulation and 
instructional subsystems, each specifically 
designed, for a vastly different punpose. 
The instructional subsystem encompasses 
capabilities . that directly support the 
training process. These capabilities focus 
on the instructor, the . trainee, and 
training 1 system management. Their 
incorporation in modern s training 
devices/training systems is., essential to 
achieve the high levels of training 
cost/effectiveness requisite for adequate 
operational effectiveness of modern weapon 
systems. 

Although many training support 'capabilities 
should be automated, such as the,, computer 
driven automatic interactive target and 
others discussed in . this paper, many 
additional capabil ities that are part of 
the instructional subsystem can be 
effectively accomplished via traditional 
manual means (e ,g. , instructor guidel ines 
for exercise development). The training 
device/training system should provide a 
range of capabilities in the instructional 
subsystem Selected on the basis of their 
cost/effectiveness impact on the training 
process. These capabil ities** furthermore, 
should be flexibly tailored to each 
specific training situation. The training 
device/training system can greatly assist 
the instructor in enabling him access to 
here-t6-fore unavailable complex 

information directly impacting ' trainee 
performance (e,g,, probability of 
counterdetection). Presentation of this 
\ complex information can also m be enhanced 
via presentation on highly effective media 
(e,g,, large screen' graphical display), A 
variety of data manipulation capabil ities 
can enable disecting various operational*, 
problems, investigation of alternative 



actions, illustration of- a r^nge of 
examples, and so on (e.g.* vi^fast-time 
computer modeling). These and f many other 
computer-based and manual capabilities make 
up essential elements of "the instructional 
subsystem. 

The instructional subsystem encompasses 
elements beyond those discussed in this 
paper, such as a variety of manual media 
typically used in the trairting process. 
The instructor himself, for example, is 
probably the single most important element 
'of the instructional subsystem and the 
training system (Gardenier and Hamuli, 
1981). Empirical research ha*s shown that 
the Instructor can have a substantial 
impact on the effectiveness ,of 

-.simulator-based training*, The capabilities 
discussed in this papqs&provide . tools for 
the instructor to enabTCr him ,to achieve a 

, more effective training process. 
Nevertheless, the fundamental 

characteristics of the * instructor himself, 
independent of these tools, may have a 
substantial impact on the effectiveness of 
any training situation. ^ * 

As discussed early in this* paper, a^e 
now emerging on the third generation 
training device, emphasizing the. importance 
of the instructional subsystem and built-in 
training technology capabilities to enhance 
the training process.. ^Although -the 
emphasis given to the instructional 
subsystem in the past has been relatively 
minor^ its importance 'is. becoming 
recognized,' with increasing emphasis given 
to these capabilities in the design of 
today's training devices/training systems. 
* SftARTTS is one example, and a forerunner of 
that trend. Another example is a set of 
guidelines for' deck officer training 
systems (Gynther, "Hammell, Grasso, and 
Pittsley, 1982) which address the use and 
design of the shiphandliog/ship bridge 
simulator for training senior commercial 
ship deck officers.*' This report recognizes 
and places emphasis on -"three major 
elements of the training system - the 
simulator design, the training prograifi 

-Struct u re^ and _Jthe_. __ Just jcjjclfir_ 

qualifications" (page i). This document, 
which* Ts intended to provide training 
system design guidance to the potential 
users of shiphandling/ship bridge 
simulator-based training devices, addresses 
and recommends many of the training - 
assistance characteristics discussed in 
this paper, and embodied" in the SMART TS 
preprototype. 

The instructional subsystem has been 
presented- in this paper as one Of two major 
parts of the training device. As such the 
training-related capabilities were strongly 
emphasized, 'while the fundamental 



engineering aspects of this subsystem were 
played down* In the past, along* with >the 
1 ack -of adequate instructional 
cfcpabil 1 ties,* . most training* devices 
received inadequate attention and scrutiny 
of their training-related capabilities. 
Following from the effective procedures 
established to monitor the design ^ and 
development of the hardware and, software 
aspects of the training device, a- similar 
procedural ^policy should be established by 
training device procurement activities to 
monitor and evaluate the potential training 

f effectiveness of the training device. 

r" These procedures should evaluate' the 
training device from its initial conceptual 
design stages on through development, 
factory acceptance • testing, and site 
acceptance testing. The focus* of^these 
additional evaluations should not be on the 
specific hardware and software engineering 
aspects, but rather on the potential impact 
of the trajners and their development 
characteristics on training effectiveness. 
% The evaluation of the potential training 
devices characteristics should r*6t end with 
the initial conceptual design; rather, it 
should continue to monitor the specific 
details of the implementation of those 
initial concepts. For example, a detailed 
human factors review of the various 
characteristics of the instructor interface 
should be accomplished at appropriate 
stages during the ' development process to 
verify effective design -details. ' SMARTTS 
developed a training system evaluation test 
(TSET) to be conducted at the time of the 
Factory Acceptance Test for the hardware 
and software. The TSET addressecf specific 
characteristics ,of the design as they were 
expected to impact the effectiveness of the 
training process. This type of evaluation 
.should* take place at all stages from 
initial conceptual design through' the 
production run of a particular device, and 
continue after the device is operational to 
constantly evolve its capabilities and 
hence improve its training effectiveness. 

This paper focused heavily on training 
system aids for assisting the instructor, 

and pro vi di nq an ef f e ct i ve interface with 

the trainee. It did address, although to a 
limited extent, the /importance of the 
training methodologies employed for 
achievement . of an effective training 
process. The instructor, the training 
deVice and its tools are mechanisms for 
implementing an effective training 
pr^cess.^^ The methods they use in * 
implement irig^ this process are extremely, 
important for achieving maximally 
cost /effective training. The training 
methods are, obviously, *an extremely 
important part of the instructional 
subsystem. They represent the strategy 
that is employed to achieve the training 
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objectives. As training system capabilites 
improve, the breath and potential 
effectiveness of available training 
methodologies also increase. For example, 
learning by example has been shown to be an 
effective 'training method (Lesgold et al, 
1978). This can be accomplished in the 
modern training system in a variety of 
ways, including several facilitated by 
capabilities discussed in this paper (e.g., 
fast-time modeling in the . classroom to 
investigate alternative tactics). By 
providing appropriate instructional support 
capabilities on the training device, 
effective training methods that could not 
be used due to the problem complexity or 
other situation limitations are now 
available. 

The training expert must also be careful 
not to overlook other training methodology 
principles which are available but might be 
overshadowed by tfr\e automated capabilities 
provided on sophisticated training - 
devices. For example, overlearning is a 
potentially effective approach for the 
training o& individuals to perform under 
high stress conditions; to take advantage 
of the benefits of overlearning in such 
situations does not require 'any particular 
instructional subsystem capabilities 
(Fitts, 1965). As always, the most 
cost/effective training process will be 
achieved by taking into account all of the 
relevant elements that impact that process, 
including the simulation characteristics of 
the training device, the training methods 
employed, and the capabilities of the 
instructional subsystem. 
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ABSTRACT 

As the quality of cockpit simulation increases and as economic pressures 
force more training into simulators, the number and type of training tasks 
handled in tile simulator are also expanding* Many of these new tasks are 
procedures oriented. This is particularly evident in pilot replacement training 
in high performance aircraft* In such situations the simulator is us€d to 
expose the student to emergencies and other situations where judgement and 
the ability to follow certain procedures accurately and quickly are vital. 
To do this rational techniques are needed to detect and assign meaning to 
the procedural events. This paper yill relate in some depth the efforts 
to incorporate a comprehensive procedures monitoring and scoring facility 
in an Operational Flight Trainer. 



BACKGROUND 

The quality of flight and weapons systems 
simulation in military simulators has advanced 
rapidly with the application of digital com- 
puters. This is particularly visible in 
the areas of visual scene and weapons system 
simulations » Transitional (from one aircraft 
to another) training curricula have taken 
, advantage of these recent-, advancements and 
flight trainers are now being used in scenarios 
where more than just flying skills and the 
rote responses to emergencies are being taught. 
Instructors are now subjecting the student 
to situations where pre-planning, headwpxk 
and a thorough knowledge of the aircraft 
and its systems are required. 

General Problem 

The quality of training received from 
these expensive systems depends a great deal 
on the instructor's ability to operate these 
trainers and of his ability to keep up with 
activities in the cockpit. Often the existence 
of such ability is an exception rather than 
the rule. This is not because instructors 
are incapable of the tasks, but because the 
complexities of operation require substantial 
training and experience in order for the 
instructors to become proficient. And operation 
of the trainer is 6nTy one of the many respons- 
ibilities of the flight instructor. 

An example of such a situation stems 
from a recent improvement to the Navy's F-14 
Operational Flight Trainer (OFT), device 
2F95. The visual system has been enhanced 
to simulate carried operations including 
catapult launches. The training syllabus 
has taken advantage of this and now single 
engine failures are being practiced .during 
the catapult shots. A timing problem* exists 



in the". insertion of the engine failure after 
the catapult stroke has started. If it is 
not inserted at precisely the correct moment, 
the simulation is not accomplished correctly 
and the training value is lost. Therefore 
the instructor spends most of his attention 
in the operation of the trainer and not to 
the student's reaction to the emergency. If 
the student crashes, the instructor obviously 
notices the result, but does not know what,* 
if anything, the student was doing prior 
to getting wet. 

Recent emphasis has been placed on 
simulator design' to help the instructor with 
scenario generation and problem control. One 
of the important aspects- of problem control 
is that of procedures monitoring and performance 
measurement. Though simulators in the past 
have attempted to provide the instructor 
with this type of information, in most cases 
' these enhancements are the first to be ignored 
by the instructor. 

Early Attempts 

An example of recent performance measurement 
capabilities are on the existing F-14 OFT. 
This trainer has the capability of monitoring 
the performance of a trainee and issuing 
a hard-copy printout whenever performance 
limits are exceeded. The instructor may 
select five of ten flight parameters and 
set a low and high limit on these values. 
He may also segment the exercise -by time. 
Then by paging into an alphanumeric display, 
he may observe the trainee performance with 
respect to theste flight parameters. 
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Some of the problems voiced by instructors 
with respect to this particular mechanism 4 



are: 
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The performance monitoring is restricted 
to flight parameter! only. 

The hig h and low limits are not standardized 
and the values are often meaningless 
because they usually must be related 
to the scenario.- 

Indexing by time into the training 
session is not effective in that instructors 
do not keep track of a training session 
by time, but by training task. Time 
references may be meaningful if used 
within^a particular training task but 
not iff used with respect to the entire 
training session (e.g. two minu/es 
after commencing. an approach is meaningful, 
however, 17 minutes after ,commenc ing 
the exercise is not). 

Setting up the performance -measurement 
is 'a laborious process. The instructor 
must use a keyboard and paging functions 
for data entry. 

The data generated is not readily available 
nor is it in a format easy to interpret 
and apply to the evaluation of the 
student's progress. 

Another performance measurement feature 
in a recently delivered flight simulator 
is an overview of the cockpit activity where 
the trainee's 20 most recent actions are 
listed on a CRT. 

Interviews w,ith and observations of 
users of this device indicate that the time 
event monitor is not very effective for monitoring 
specific procedures' in the cockpit because: 

Not all of the 20 most recent actions 
may be applicable to the task the instructor 
is monitoring and the page tends to 
become cluttered with inappropriate 
information. 

Certain procedures applicable to the 
task may not be represented on this 
* page and must be followed on another 
area of the Instructor/Operator Station 
(IOS). 

When the' student becomes very busy 
in the cockpit, these action messages 
appear and disappear more qurckly than 
they can be read. 

^ ISS APPROACH 

Rational techniques are needed to detect 
and assign meaning to the procedural, events 
and performance measurements in' a trainer. 
This information must be made readilpavailable 
and must be presented in an easy to /understand 
format to the user. While this ma ; y not seem 
to be a difficult task on the surface it 
has not been built into any trainer Successfully. 



The remainder of this paper describes 
efforts by Logicon personnel to incorporate 
a comprehensive procedures monitoring and 
scoring facility in the Instructor Support 
System (ISS). The ISS is the product of 
a general effort to improve IOSs sponsored 
by the U.S. Naval Training Equipment Center. 

Problem Management 

In approaching this effort one of^ the 
first technical issues that arose was the 
sheer size of the problem and computer processing 
power required to handle it. The F-14 OFT 
has approximately 350 switches and other 
pilot adjustable mechanisms in the, cockpit. 
If one attempted to monitor all of them all 
the time and condense, evaluate, and present, 
derived information to*the instructor, one 
* would require moire processing power than 
is available with any computer and more software 
than a small army of talented programmers 
could produce. 

The only reasonable solution appeared 
to be the creation of a mechanism whereby 
only the subset of activity in the cockpit 
that was appropriate at the moment would 
'be monitored and processed. This resulted 
in the establishment of "task modules 11 .* 

These are operationally logical training 
segments such as takeoff, departure, or hydraulic 
malfunction. A training session/ scenario 
is composed of a group of these task modules. 
Part of each task module are detectable events . 
The ISS can follow the scenario by watching 
for these events. For example, a combination 
of events su4h aa gear transition to up, 
flaps tranaition to up, speed increaaing 
through 200 knots, heading increasing and 
passing 275 degrees, "starta" a task module 
representing a particular departure from 
NAS Miramar. When this task module is started, 
only those flight parameters yd procedures 
applicable to this training task are monitored 
and processed. 

Task modules may'also be "forced" to 
run by direct action of the instructor. 
More than one task module can run at the 
same time, if the activities associated with 
them overlap. This quality was also used 
to monitor vital safety functions at all 
times the aircraft was off the ground. A 
.task module was defined whose start condition 



■ La SK. UUUUic wao «v-*^."v-- . . 

was "weight off wheels" and stop condition 



*A companion paper, also prepared for this 
workshop, entitled MODULAR CONTROL OF SIMULATORS 
explains the concept of task modules in depth. 
Although this paper indicates how task modules 
fit into procedures monitoring and scoring, 
the reader is encouraged to refer to the 
other paper for additional information. 
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was "weight on wheels". Among other items 
it monitored the^ accelerometer and informed 
the instructor of any aircraft overttresses . 
It was nicknamed the "umbrella" and kept 
the l ISS from looking stupid by not informing 
the instructor if some ditaster occurred 
The maximum number of concurrently running 
task modules in the ISS is four. This constraint 
was due to processing power limitations, 
but it did not hamper operations. 
Task Module Structure 

In order to understand in more detail 
how the ISS monitors procedures and performance 
. measurement, one must examine the ISS software 
and task module structure. The system is 
driven by a data base consisting of task 
module data files. Each task module is defined 
by a set of data files which are unique to 
that individual task. 

One of the files within a task module 
set defines detectable events (or actions) 
that are directly, functionally, and physically 
related to the training task. Directly related 
actions are those that are depicted in the 
NATOPS manual as part of the procedure (e.g. 
switch A - off, switch B - stby, lever C 
- up, etc.). Functionally related actions 
are any actions which are similar in function 
but are not part of the procedure, (e.g. 
in a particular hydraulic failure a secondary 
hydraulic isolate switch is part of the hydraulic 
system of the aircraft, but does not play 
a^ part in the particular procedure.) By 
identifying this switch, a diagnostic message 
can be created informing the instructor if 
this switch is mistakenly thrown. Physically 
related actions are those linked to switches 
that_are in close proximity to the appro- 
priate switch and ma^ look the same (e.g. if 
the wrong circuit breaker was pulled in a 
procedure, this identification would produce 
a diagnostic message). 

In the definition of these events the 
following considerations are made; 

Relationship of the cockpit device 
value to a reference value (greater 
than, less than, etc.). 

Stable time (to avoid triggering on 
transient changes). 

Whether the event is external, something 
happening in the trainer,' or internal, 
*a combination of external events. 

Whether the event is a verification 
of a value or a change in its state. 

Whether the event has the "dropout" 
property (whether it it considered 
to be another occurrence when the reverse 
of the initial happens). 



Procedures Monitoring 

A program dedicated to monitoring these 
events is notified when a task module becomes 
active and opens a file defining the appropriate 
events for that task module. In this way, 
only a select group of switches are monitored 
at any one time. 

Whenever any of these events have been 
detected, this event monitoring program informs 
another program, dedicated to the procedures 
monitoring function. Like the event monitor, 
this program is also informed when a particular 
task module becomes active. It opens a file 
which has defined the relationship of the 
events • 

This program evaluates 'action* with 
respect to relationships and contingencies 
and causes- a variety of actions to be taken 
by the system, fhat is, it can evaluate 
a sequence of events such as the case where 
switches ABC must be thrown in sequence, 
and C D E must be thrown/after the the first 
three but not necessarily i n sequence. An 
example of this is the engagement of nose 
wheel steering during Un F-14 takeoff. The 
steering is optional prior to 15 knots, a 
requirement between 15 and 80 knots, and 
must be disengaged prior to 100 knots. 

With respect to contingencies, not 
only must the specific action be identified, 
but also the circumstances under which it 
is to take place must be defined. That is, 
when an event occurs, alternate actions may 
be specified depending on other circumstances 
or contingencies. For example, during takeoff 
if the aircraft heading drifts off the runway 
heading, by a specified amount while the plane 
is still on the runway, an illegal action 
has occurred and a diagnostic message is gener- 
ated. However, heading changes after the 
wheels are* off the runway are normal and 
no action is taken by the system. 

Performance Measurement 

0n_e of the actions that may be taken 
by the system as the result of the occurrance 
of an event is taking measurements of flight 
parameters. For-example, when the aircraft 
reaches a certain altitude the system may 
start measuring the pilots ability to maintain 
that level altitude. Or it may take a single 
measurement at a specific point. For example, 
when the aircraft reaches Ground Controlled 
Approach (GCA) minimums, it may measure the 
distance from the centerline of the runway. 
A performance measurement program is dedicated 
to taking measurements associated with a 
task module while it is active and when completed,- 
calculates a score for the task. It, like 
the other programs, is informed when a particular 
task module is active. 
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The measurements are each defined in 
one of the task module files. They fall 
into several general catagories: 

Continuous. A measurement with predefined 
start and stop conditions. The parameter 
is sampled 1 to 20 times per second. 

Monitor, A continuous measurement 
wherein a limit is defined. The parameter 
is sample during the time that it is 
outside the limit. r, 

Snap Shot. A single parameter is sampled 
once. 

Procedures Evaluation 

Performance measurement for normal and * emergency 
procedures, because of- their nature, is handled 
quite differently than flight parameters. 
The following factors, or subset of them, 
are considered: 

Critical errors. Those actions which, 
by their omission or commission, will 
produce catastrophe* 

Recognition reaction time. This measures 
student response to certain cockpit 
stimuli in the case of aircraft system 
degradation and/or failures. 

Total procedure time. This measures 
the total time required to complete 
the procedure. 

Percent of mandatory actions taken. 
These are actions that are depicted 
in the F-14 NATOPS manual. 

Percent of optional actions. These 
are actions that are not manda-tory 
but demonstrate that the student is 
well prepared and well in command of 
the situation. 

Scoring 

The performance measurement program 
is informed when the task module is completed 
and calculates a score. It does so by comparing 
both the flight parameters measured and the 
procedures evaluated with criteria unique 
to the task module. 

Scoring of performance by a computer 
is a controversial issue and is. one of the 
first to be questioned by students and instructors 
alike. Developers of the ISS have attempted 
to defuse this problem by putting the grading 
criteria under control of the user's community 
and by making visible to the instructor Cat 
his request) the measurements and calculations 
used to derive the score. 

The key to this is a scoring template 
in which are placed all measurement definitions 



and scosing formulas. This template is one 
of the files making up the task module definition 
)and is easily established or modified using 
a text editor. After a ta«sk module has been 
executed the system display* the template 
in a format similar to that in which it was 
created but with the addition of the actual 
measurements taken and all the figures used 
in deriving the final score. 

Figure 1 is a template for a task module 
which defines thVSAN. PEDRO DEPARTURE from 
NAS Miramar. The first column under "MEASURE" 
are measurements that the student is gr/d^d 
on in performing this departure. 

DME SNAPSHOT is the distance from a 
particular f ix ^the student is to fly over. 
This is a single measurement. 

RADIAL DEVIATION is the ability t/ 
fly a specific radial. This is a continuous 
measurement. RMS indicates that "root mean 
squared" is the basic transform used in processing 
the data. 

AIRSPEED DEVIATION is ability to maintain 
a specific airspeed. This is a continuous 
measurement and the RMS transform is used. 

The second column under "NOMINAL" is 
the reference value of the particular measure- 
ment. This can also be used to differentiate 
between measurements with the same name. In 
this example the first radial measured is 
the 280 degree radial and the second is the 
300 degree radial. Likewise the 1, 2, 3, 
next to the DME SNAPSHOT measurements indicate 
the first second and third fixes in the departure. 

The "4.0, 3.5, 3.0, 2.5" columns are the 
scores on the Navy's 4.0 scale. The grade 
for a particular step in a task module is 
derived by comparing the measured value with 
the range of values below these columns. 
For example, the first measurement listed 
on figure 1 is the distance at closest point 
of approach from the first fix the student 
was to fly over. The actual value measured 
was .79 miles recorded under the "VALUE" 
column. By looking across the template opposite"" 
the first DME measurement we see that the 
measured value, .79, falls between 0 and 
2.5 which is under the 4.0 column. Therefore 
the student is assigned a 4.0 for that measure- 
ment. This is recorded under the "GRADE 
column/ The last column "WT" is the weight 
factor given to that particular measurement. 
In this case it was worth 10 percent of the 
total grade. After all of the steps have 
been calculated in the method described above, 
they are summed to form the final grade, 
which in this example is 2.7. 

This example does not show any procedures 
scoring. If it did, REACTION TIME, % MANDATORY 
STEPS, or the like would appear as additional 
measurements. That is, procedures are scored 
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in the same manner as flight parameters. 
An exception is the CRITICAL ERROR . If one 
of these occurs, the task module is assigned 
the^ lowest possible grade regardless of other 
measurements . 

Information Presentation 

Although it may not seem to have direct 
application to procedure monitoring, the 
manner in which the information is presented 
to the user is very important to the successful 
application of these features. The ISS approach 
again centers around task modules. As part 
of each task module's definition, pictures 
are defined in the task module data files. 
As task modules become active and are completed, 
appropriate pictures are added to and removed 
from the user's displays. Task modules covering 
procedures (e.g. check lists, malfunctions) 
are composed of text identical to the procedures 
as depicted in the aircraft dNATOPS manual. 
As each step in the procedure is accomplished, 
the action is noted next to the appropriate 
step. 

Figure 2 shows two concurrently running 
task modules on the upper half of the display, 
a normal checklist on the left and a malfunction 
on the right. Both are procedure oriented 
and indicate that the steps are in' progress. 
The section below, labeled "diagnostics" , 
containsmessages generated by the procedures 
monitor program* These appear at the bottom 
of that section and move up either when displaced 
by a new diagnostic message or when 30 seconds 



have passed. All messages eventually disappear 
from the top of that section. The small 
rectangles represent menu selections that 
can be nicked by the user yi^a touch mechanism 
as8oci 4 at.ed..vitfr'trtie i' 'display . This is the 
in£ut/control mechanism of the ISS. 

SUMMARY 

This paper has described a software/hardware 
system that is able to monitor procedures, 
and score individually, and concurrently, 
-the many training tasks in a complex training 
scenario. This system has approached the 
problems from the instructor's viewpoint 
and is* intended to give him understandable 
and appropriate information on an uncluttered 
instructor/operator station. Using this 
he will be able to better evaluate the student's 
performance and provide the instruction needed 
to improve it. 

The ISS, of which this procedure monitoring 
and scoring mechanism is a key part, has 
been under development for several years 
and is now undergoing operational evaluation 
at NAS Miramar. The concepts developed in 
this prototype promise improved training 
in existing flight trainers and can serve 
as the basis for instructor/operator stations 
on new simulators* 

ABOUT THE AUTHORS 
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ABSTRACT 

The conventional use of instructor pilots at flight simulators contains a 
number of problems— a continuous training program is required because of instructor 
rotation to the assignments, pilots are often not interested in being simulator 
instructors, and they frequently require assistance from device operators. Further- 
more, to obtain maximum utilization of a "simulator a large number of instructors 
are required. The use of student self-training programs, enabling a simulator to 
be operated without an instructor when required, would solve many of these 
problems. This paper presents the advantages of such a capability and discusses 
the related design considerations. It describes the required instructional 
programs and offers several solutions to the problem of locating simulator con- 
trols in the cockpit, in addition, it discussesTthe subjects of safety and 
realism and concludes with recommendations applicable to future specifications. 



INTR0DUCTION 

Conventionally, flight simulators have 
been equipped with more or less elaborate 
instructor stations, located either on-board 
(in the cabin behind the pilot's or 
co-pilot's seat) or remotely (entirely 
removed from the cockpit). With the 
on-board instructor station the instructor 
is able directly to observe the crew 
members and evaluate their performance; 
.with the remote instructor station the 
instructor observes the crew performance 
through repeater instruments and CRT displays. 

In either case, the trend has been to . 
strengthen and emphasize the role of the 
instructor. Instructor stations have been 
designed to enable the 'instructor to take 
maximum advantage of the simulator's in- 
structional capabilities, to provide him 
with extensive information- regarding the 
progress of the student's training, and at 
the same time to reduce his (the instruc- 
tor's) workload. 

Yet there are many problems inherent, 
in this arrangement. Instructors require 
training--the more sophisticated the 
simulator the more extensive is this 
requirement. Then, after a certain amount 
of utilization, most instructors rotate to 
other assignments and never return to 
instructing on the simulator. Or, if they 
go to sea duty or other forms of lengthy 
TOY (temporary duty), they need retraining 
when they return. 

Many pilots do not want to be simulator 
instructors. If the simulator is complex, 
pilots find that learning and keeping 



current on its operating procedures de- 
tract from their concentration on keeping 
current on the aircraft. These procedures 
are perhaps not difficult to learn but 
apparently are impossible to retain with- 
out frequent practice. The problem is 
exacerbated by the fact that toany instruc- 
tor stations are equipped with a confusing 
. array of switches and other controls, often 
pdbrly labeled, and offer a bewildering 
variety of* instructional features, many 
rarely used. Furthermore, most pilots 
never accept having to operate an alpha- - 
numeric keyboard with lengthy formats 
for data entry. 

It is possible to design instructor 
station controls, displays, and programs 

-so as to reduce the demands on instructors. 

"The instructor station for Device 2F132, 
the F/A-18 Operational Flight Trainer^ 
is an example of such an effort. U) However, 
the ultimate solution tp this requirement 
has not yet been demonstrated. 

The use of device operators is a way 
to solve these problems. Device operators 
who do not have other, conflicting duties 
can keep current on all of the instructor 
station features and procedures and can 
assist instructors in many ways, from 
serving as operator of the alphanumeric 
keyboard to handling all of the complex 
instructional features. However, device 
operators are usually not provided at 
on-board instructor stations, probably in 
order to minimize the number of peesons* 
in thej:a6in and to conserve on the size 
of the instructor station. Furthermore, 
if device operators are always required, 
in addition to instructors, the manpower 
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requirements for operating the simulator 
become significantly increased. 

Another approach is to hire former 
pil 0 ts--retired military or naval personnel— 
to be simulator instructors. Since they will 
presumably have no other function, they can * 
keep as proficient on the simulator as 
device operators can. However, there are 
issues involved in this approach— the 
instructors 1 credibility with the students, 
the cost of the additional personnel, and 
the availability of qualified individual s~ 
that may militate, against widespread 
adoption. 

Regardless of the source of the instruc- 
tors, the requirement to have the simulator 
manned by someone other .than the students 
is a limitation on its utilization. A 
modern 'simulator" is capable of operating 
at least 16 hours per day. To continually 
provide an instructor for this level of 
operation for five days per week requires a 
force of at least six instructors if they 
are not active duty pilots, or approximately 
twice that number if they are. 

A different solution to all of these 
problems is to develops concept of student 
self-training^rnot to eliminate aM instruc- 
tors, of course, but to provide users with 
the capability to conduct some training 
without the presence of an instructor 
-being required. 

CONCEPT 

In a simulator with a student self- 
training capability the student should be 
able to conduct a complete training exercise 
using controls entirely in- the cockpit. 
To this end he should be able to initialize 
at a desired geographical location, fly a 
pre-planned flight profile; introduce and 
remove malfunctions, operate simulator 
control functions such as freeze and crash 
override, and operate instructional 
features such as demonstrations and per- 
formance measuring. 

The exercise should be part of an 
overall training syllabus that would include 
both aircraft missions and simulator^exer- 
xises. The r\umber of self-training exercises 
with ah instructor -being present should be 
planned in advance and scheduled in the 
syllabus. Self-training should never be an 
"ad lib" use of the simulator resorted to 
only when an instructor is absent. 

The optimum ratio between self-training 
and instructor-monitored exercises in 3 
syllabus would depe_nd on a variety of condi- 




tions. The availability of trained instruc- 
tors would be a major factor. If the number 
of available instructors is small and the 
number of students is'large, self-training 
exercises would comprise a substantial part 
of the syllabus. The qualifications of ' 
the students would be another factor. A 
pilot who needs only refresher training 
could use more self-training exercises than 
a recent flying school graduate. In general, 
if instructors are used -primarily for 
initial indoctrination and/for checkrldes, 
the ratio of self-training exercises to 
instructor-monitored exercises could be 
in the order of four or five to*one. 

DESIGN CONSIDERATIONS 

providing a student self-training 
capability will have a significant impact 
on simulator design--on the instructional 
system software and, to a lesser extent, 
on the student station hardware (i.e., 
the cockpit). 

Controls 

Controls in the student statJfn will 
have to be considerably more elaborate than 
for a conventional simulator, which may 
have only a Freeze switch located tfery 
' inconspicuously. How elaborate will depend 
on the number of functions provided for the 
student— to the same degree that the com- 
plexity of an instructor station depends 
on the functions provided for the instructor. 
If cost considerations dictate an austere 
approach, student station controls can be 
kept to a minimOm— only sufficient for 
initialization*, freeze, motion control and 
emergency off . Additional capabilities 
can be provided at increased cost, although 
the fact that these capabilities must be /in 
the cockpit, not at an instructor station, 
will have a distinctly limiting effect. 

-* Student station controls must have 
certain required characteristics. The 
first of these requirements is that the 
ftntrols— and the instructional features 
that they relate to--must be understandable 
and easily operated by untrained individuals. 
In this cas.e, "untrained" means not having 
attended an operator's course and having 
been required to read only minimum instruc- 
tions on the simulator's operations. 
Simplicity and consistency in tfie design of 
simulator controls will be required to a 
degree far greater than heretofore. 

* A- second requirement is that student 
"station simulator controls be- easily 
accessible. Since the student will be 
devoting his full attention 'to flying the 
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simulator, It Is Imperative that the distrac- 
tion of locating and operating non-atrcraft 
related controls be kept to a minimum. 

A third requirement, which is not easily 
reconciled with the previous one, is that 
controls [>e located as Inconspicuously as 
possible. On the premise that student accep- 
tance of a simulator depends on how closely 
Its cockpit, sound effects, and "feel" resem- 
ble the actual aircraft, it is apparent that 
panels, switches, and indicators that are 
foreign to the aircraft would be an irritant 
to the student. Obtrusive simulator control's 
in the cockpit would be as distracting as 
those that are hard to find. 



Two siipulatoVs built by Sperry Systems 
Management - SECOR contain^controls for 
operation from tfie cockpit^ although -how 
much they wil.l be used> for aircrew self- 
training has apparently not been determined 
The two approaches have both similarities 
and differences, with respect to scope of 
capabilities and concept of operation. 

One design, the CH-53 Operational Flfght 
Trainer, the first unit of which is scheduled 
to be delivered to the U.S. Marine Corps in 
late summer 1982, contains a small control 
panel on each side of the cockpit, above and 
in front of the two side windows (see 
Figure 1). The panels are identical, each 
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Figure 1. CH-53 OFT Remote Control Panels 
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having push-button switches for Initial 
Conditions Select, Freeze, Motion Ready/On, 
Emergency Off, and Motion Emergency Disable. 
Two thumbwheel- controls on each panel 
enable the students to select a set of 
initial conditions from twenty available. 
The control panels are not covered or 
otherwise concealed but are located above 
the pilots and copilot's normal forward 
field of view. „ 

The other approach is found in the 
HU-25A and HH-65A Flight Training Systems 
currently being built for the U.S. Coast ♦ 
Guard for delivery in early 1984. A 20-key 
keypad, provided primarily for "the instructor 
station, *which is located in the cabin, » is 
installed with a cable that enables it to 
be placed on the pedestal or the floor 
between the pilot's and copilot's seats. 
The keys include. Freeze, Override (for 
crash), Store (for flight conditions), Reset, 
Enter, Clear, Remove (for malfunctions), 
ten digits, two punctuation marks, and 
backspace (see Figure 2). 
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Figure 2. HU-25A/HH-.65A FTS Keypad 

In both the CH-53 and the Coast Guard 
trainers the students will be able to use 
the instructor station controls to accomplish 
functions that are not possible via the 
cockpit simulator controls. Conducting 
demonstrations would be an example of such 
a function. Also, if necessary one of the 
students will be able to call up displays on 



the instructor station CRTs and use ttjem for 
sel f-tr4jn1ng purposes. For example, the 
students' will be able to inspect the air- 
craft track depicted on an approach display, 
after making an approach* 

The self-training capabilities of these 
simulators .are facilitated by the fact that 
they have on-board instructor stations and 
two-man crews. The copilot -can readily go . 
to the instructor station and use the simu- 
lator control s as he desires without- 
interfering with what the pilot is doing. 
If the simulator had a single-place cockpit 
and a remote instructor station, the 
student station controls for self-training 
would have to be more self-sufficient. - 

Formats 

The design of the controls will be 
influenced considerably by whether the data 
entry formats are page-dependent or not. 
For a "worst case" situation, in which the 
student will have no access to the display 
system, the formats will have to be non-page 
dependent, utilizing input codes for entering 
and clearing malfunctions, modifying flight 
parameters such as fuel load and armament, 
and modifying environmental parameters such 
as ceiling and visibility. 

Since an alphanumeric keyboard in the 
/Cockpit is out of the question, all input 
codes will have to be useable on a keypad. 
This requirement suggests that the codes 
should be numeric, but to help the student 
remember at least the most-used ones the 
keys on the keypad could b£ labeled both 
alphabetically and numerically, like a 
telephone, and the codes could be alpha- 
betical . 

If alphabetical codes are used, and 
other alphabetical inputs sucli as N, E, S, 
and W for latitude and longitude are 
required, the student will need a shift 
key to select one of the three letters on a 
key and to differentiate between letters and 
numerals. This 'approach tends to complicate 
the operation of the keypad but the alterna- 
tive, the use of all -numeric codes, is less 
desirable. 

There is another alternative to the 
u£e of shift keys, i.e. a two-digit code', 
used inthe FAA's Voice Response System for 
telephonic weather briefings, that identi- 
fies each letter on a key. For example, thfe 
letter "A" is selected by entering "2,1" 
(referring to the 2 key and the first letter 
on it). Similarly, "B" is "2,2". This 
method, however, will increase the time 
spent in entering all formats', and k is con- 
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sidered to be acceptable pnly if the Input 
codes can be kept to one or two letters in 
length. 

The number of input codes' required will 
depend on how much capability is provided to 
the student. It is likely, however, that 
the number Vill -exceed what can be remem- 
bered by the student, particularly if he' 
is not using the simulator frequently. 
Therefore, a list of input codes will be 
1 needed to be kept in the cockpit. This list 
could be part of an abbreviated checklist, 
similar to the Pocket Checklist used in 
aircraft, and could cover all of the oper- 
ating procedures that the student would need 
to be reminded of. 

Functions 

In a self-training mode the student 
could forego' many functions that are" 
available at the instructor station in a 
conventional simulator. Also, many func- 
tions that are usually accomplished with 
controls could be performed via alphanumeri- 
cal entries to the computer. These 
principles will have to be imaginatively 
applied to the design of the student station 
in order to keep the number of controls to a 
minimum. A minimum-sized control unit, 
whether it is a panel like the CH-53 OFT or 
. a keypad like the Coast Guard simulators, 
is essential to meeting the three required 
design characteristics discussed previously. 

v _ • 

The following discusses ^ broad fan^P, 
of functions typically exercised at an ^ 
instructor station and describes how they * ; 
could be best performed with- controls in 
the cockpit. 

Turn-On Procedures. Turn-on procedures 
can be considered to include powering-up, the 
system, initializing the computer and 
peripheral equipment, turning on the visual 
system if the>re is one, and performing a 
readiness test. None of these procedures 
requires special consideration for the self-* 
training concept; they can all be accom- 
plished by maintenance personnel, who 
presumably will always be present. 

Initial ization. In conventional simu- 
lators, initialization is accomplished in a 
variety of ways: thumbwheel controls, 
pushbuttqn switches, and alphanumeric key- 
board or keypad entries are used, sometimes 
in combinations, to select the set desired 
and enter it into the computer. Keyboard 
entries and pushbutton switches are used to 
display the available sets on a CRT at the 
instructor station, if the instructor 
wishes, he can modify the parameters- by 



making keyboard entries. 

For -the self-training concept, keypad 
entries should be used exclusively „ in order 
to minimize the need for hardware controls. 
In the absence of a CRT in the, cockpit, the 
available sets can be included in the abbre- 
viated checklist mentioned previously. ^ 

The question arises whether the student 
. should be allowed to make on-1 fne modifica- 
tions to an initial conditions set before it 
is entered, and if he is afflowed to make * 
« modifications, how he will be able to record 
them. The simplest approach is to allow no 
pre-entry modifications'. In this case, if* 
the student is not satisfied with the initial 
.conditions he can make parameter changes 
after the set is entered. The student will 
then observe the results directly on tfee 
aircraft instruments. 

Motion. Because of safety considera- 
tions, the motion system presents a special 
problem for the ^elf-training concept. With 
a six-post system, the operator, before 
activating the system, must have a positive 
indication that access ladders are stowed 
* and maintenance platforms around the cockpit- 
are cleared. Even smaller systems have 
safety problems . 

One approach, for the self-training 
concept, would be to make the maintenance 
personnel responsible for operating the ' 
motion system— they could use an intercom 
system to communicate with the student when* 
necessary. On the other hand, it would be 
almost mandatory to provide a "disable" 
capability in 'the cockpit. If a switch is 
provided for that purpose, it could be - 
designed to indicate a "motion ready" condi- " 
tion and to provide. both an "enable" and 
'disable" capability for the student (in 
% some types of installation, this may not 
be feasible; an additional switch may be 
Required). Additionally, the student could 
be required to obtam^a verbal clearance 
with the maintenance personnel, over the 
intercom, before activating the system. 

Normal Operating Functions . There are 
a number of routine operating -capabil i ties , 
usually performed with push-button switches 
at e conventional instructor station, that 
must be provided in the cockpit, in af way 
that conserves hardware controls as much as 
possible. The most basic of these is ** 
Freeze; dedicating a push-button switch for 
this function appears' to be mandatory. 
Emergency OH' is a similar requirement, 
although the switch should be guarded in a 
way that guarantees argainst inadvertent actu- 
ation. C^sh Override can be accompl ished - - 
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with a keypad entry., if i*t is assumed that 
the studeofrwill not need to activate it as 

-an immediate response to difficulty. Slew- 
ing, which in a conventional ^simulator 
usually requires a joystick control ar\d a 
pushbutton switch; can be omitted in the 
self- training mode. If the student wants 
to change the geographical position of the 
simulated aircraft he can enter a new 
latitude and longitude with a keypad entry 

*(or, with a- more sophisticated program, a 
radial and DME •from a NAVAID). Or-he can 
reinitialize. Although possibly not essen- 
tial , Store IC's and Reset are very useful 
functions; they enable the instructor -to 
store the existing fligij£ conditions at any 
time and then to reset the simulated air- 
craft there when desired 1 later . If. these 
functions are provided for the sel f- training 
concept, two pushbutton switches would be 
needed. Finally;- an intercom* ca_pabil ity 
with maintenance personnel roust be provided. 
A switch is not needed, however, if the 
student's microphone is continually "hot". 

- „ Instructional Features . 'The major in- 
structional features found in most current 
simulators are demonstrations , checkrides 
(also known as programmed missions), 
playback, parameter record, and event print. 
Probably the one roost applicable to the 
self-training concept is demonstrations; # the 
possibilities for learning by observation 
and emulation, through this method, are ' 
almost limitless. All of the actions 
required to select, activate, and terminate 
a demonstration can be taken with keypad 
entries (plus use of the Freeze switch £o ^ 
start after the pre-deraonstration initial * _ 
conditions have been attained)., 

Checkrides are also very applicable to 
the self- training concept, although t.hev r _ 
will pose a problem with respect to th^ 
monitoring function. In a conventional 
simulator, the instructor uses the CRT 
displays to monitor the progress of the 
'mission; if the student makes a gross error 
sufficient to confuse 1 the computer and induce 
it. to score ji leg that is- different from the 
one that the student is flying, the instruc- 
tor can correct the situation with a. Manual 
Advance switch. Ia the self- training mode 
it is not feasible to provide the stu^gnt 
» "with this capability or to expect him to 
monitor the mission Teg by leg. It is 
essential that the mvssions be very -care- 
fully designed so as to reduce to a rainimum 
the need for instructor intervention^ if 
the mission cannot be completed as intended, 
it will have to be aborted. 

Playback (Dynamic Replay) t will be of 
lesser usefulness without;a^instructor 



.present to contribute comments. The student 
will know generally what his errors were; 
redtfing the maneuver or procedure correctly 
will be his primary concern. If desired, 
however, Playback can be operated with keypad 
entries, assisted by, the Freeze switch, 

A variation of Playback called Minute, 
Replay, or sometimes Instant Replay, enables 
the instructor to store a number of segments 
of flight history^ usually of one-minute 
duration each, and, to replay them during the 
critique period-after the mission. This , 
capability is not considered to be important 
-in the self-training mode. To provide it, 
however, a switch will be needed to enable 
the student to quickly select the segment 
to be stored. 

Parameter Record is a Tittle-used 
data-gathering capability. To operate 
Parameter Record the instructor must select 
individual parameters to be monitored, 
specify the reference value and tolerances, * 
and start and stop the recording. These are • 
time-consuming functions that are not at all 
suitable for a student to perform. 

Similarly, -Event Print is not appro- 
priate for the self-training concept: This 
feature prints, either on the Parameter 
Record printout or independently, the time 
of occurrence of a number of eyents that the 
instructor selects in advance. To be useful 
this capability must be part of a very 
detailed analysis and critique of the'mission 
which' only an instructor can conduct. 

Mai function Control . As in conventional 
simulators, malfunctions can be entered 
and cleared in the self-training mode by 
using keypad entries. The list of program- 
mable malfunctions can total several hundred 
items depending on the type of aircraft afd 
amount of attention to this subject desired 
by the user. A shorter list can be devised 
for the self-training mode and included in 
the student's abbreviated checklist. A 
' Malfunction Override control , if. desired, 
wjll requife a pushbutton switch. 

.Critics of sel f- training could say that * 
considerable trainfng value is lost, due to 
the absence of surprise, if the student 
•enters the mal function himseVf. However, 
all training value is not lost, particularly 
with respect to practicing the steps in 
emergency procedures. Furthermore, it 
would be possible to enable the student to 
program a series of malfunctions to occur, 
• randomly during a mission, thereby assuring 
that he would -at least not have foreknowledge 
of when a malfunction would occur. Also, 
programmed malfunction "packages", with 



160 



9 

ERLC 



lb 



different types of 'malfunctions and levels 
of difficulty depending on«the mission and 
student qualifications, could be entered 
by the maintenance personnel. 

Miscellaneous . Additional functions 
found in conventional simulators can be 
accommodated in the sel f -training mode with 
keypad* entries or omitted entirely in the 
interest of simplicity. A volume control 
for sound effects, for example, can be 
omitted (if considered necessary this 
function can be accomplished with a keypad 
entry). Controls for the voice recorder, 
if one is provided, can be omitted-- 
separate control is not needed in the self- 
training mode. The catapult-hold and -fire 
functions, for a carrier-based aircraft, 
can be accomplished with keypad entries. 
Alternatively, catapult fire can be program- 
med to occur a few seconds after the pilot 
gives the "ready" sjgnal by turning on his 
exterior J ights , or' in another way, after 
he lowers the hand-grip. Similarly, chocks, 
external power, starting air, rough air, and 
arresting gear' can be provided via keypad 
entries. A warning horn for a malfunction 
of the simulated oxygen system, provided 
at the instructor station in conventional 
simulators for safety reasons, can also be 
installed at the student statuon (if, in 
fact, it is truly needed) . 

Optimum Design 

In^summary, it is concluded that the 
most desirable functions for the self- 
training concept could be accomplished with 
a keypad containing 20 keys. The following 
functions anp considered to be the most 
useful: Motion Ready, Motion On, Emergency 
Off, Freeze, Malfunction Override, Store ICs, 
Reset, ten digits, Enter, Clear, punctuation, 
and backspace. 

The 20 keys could be accommodated in a 
unit similar to/the hand-held terminal manu- 
factured by the Termiflex Corporation,-* 
Various models of the Termiflex have the 
capability of displaying one or two lines of 
alphanumeric text, each 10 to 12 characters 
in length. This feature would be essential 
for the sel f-tra ining concept, to enable the 
student to verify hfr input codes and values 
before entering them into the computer. 
Three shift keys would prov-ide access to/^ 
t t"*he letters and punctuation. An example of 
a self-training terminal is shown in ~ 
Figure 3. , * ' 

The terminal should have some features 
not found in the current models of the 
Termiflex. The keys should be backlighted, 
to enable reading when the cockpit is . \- 



darkened for simulated night operation. 
Further, certain function keys (Freeze, 
Motion Ready/On, and Malfunction Override) 
should be brightened when selected, to 
provide a status indication to the student. 
Also, the Emergency Off key should be 
guarded, preferably^with a cover that would 
have to be lifted before the key is actuated. 

A hand-held terminal eliminates the 
problem of finding a blank panel in the 
cockpit behind which to conceal the switches. 
The terminal could be hung behind the seat 
or stowed under it, seat design permitting, 
and could be removed entirely for instructor- 
monitored missions if it had a plug-in 
capability similar to the Termiflex. 

Displays 

In the aforegoing discussion of simulator 
functions and the option? available for 
accomplishing them, it has been assumed that 
the CRT displays normally found at an 
instructor station would not be available 
in the cockpit. However, it is possible 
to display both graphic and alphanumeric 
data to the student via the visual system. 
This approach has been used modestly in the 
A-6E Night Carrier Landing Trainer (NCLT), 
Device 2F122; one display, depicting a plot 
of certain parameters (vertical velocity, 
bank;. pitch, rpm, glide slope, and center- 
line) recorded during an approach, can be 
shown on the two visual display CRT's in 
front of the pilot and B/N. 

In the NCLT the alphanumeric/graphic 
display replaces the visual scene; an 
alternative method is used in the A-4M OFT, 
Device 2F108/2B34F--a line of weapon scoring 
data is superimposed across the bottom of c - 
the visual scene. The parameters include a 
hit or miss evaluation, bearing and range of 
the input, airspeed at release, and others. 
This information is available only on the 
instructor station visual monitor, but it 
could be easily displayed to the piloNflf 
desired. 

The possibility of using the visual 
System for alphanumeric/graphic displays 
provides almost unlimited opportunities 
for enhancing self-training. If a side- 
viewing visual CRT is provided in the 
cockpit, it could be used, as frequently 
as desired by the sjtudent, for any of the 
djsplays available at the instructor station. 
A front-viewing visual CRT could also be, 
used, but in a more restricted way. 

Of predictable interest to t'he student 
would be the procedure monitoring displays 
of normal and'emergency procedures showing 
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the results of the student's attempts, the* 
approach displays showing the various pub- 
lished approaches and a record of the 
aircraft track, and GCA/CCA/ILS displays with 
both horizontal and vertical projections 
of the aircraft track. Static data such as 
the list of input codes and the contents 
of the initial conditions sets, which would 
be available as instructor station displays, 
could also be presented in the cockpit, 
rather than in a Pocket Checklist as 
suggested previously. Special displays 
providing checklist instructions for oper- 
ating the simulator could be included. 

If displays are available to the 
student, it is possible that he will want to 
print the most "significant ones for post- 
mission analysis. Checkride summary data 
would be particularly useful. In any event, 
the display printout function could be 
operated by a keypad entry. 

Safety 

In any discussion "of students operating 
a simulator w.ithout an instructor, concern 
is usually expressed regarding safety. 
There are a number of potential sources of 
danger around a simulator--high voltage 
electricity, high pressure hydraulics, 
the motion system, and possibly others. 
However, most of these should be of concern 
only to maintenance personnel. There should 
be no reason for students to be exposed to 
high voltage, for example. In any event, 
maintenance personnel will always be present 
during self-training periods and can be made 
responsible for enforcing restrictions on 
access to hazardous areas. 

The only source of danger of importance 
to the self-training concept is the motion 
system. The danger, of course, is to 
personnel working outsidrand in the 
vicinity of the student station. As dis- 
cussed previously, the best solution to this 
problem is to have motion system turn-on 
procedures involving both the students and 
the maintenance personnel". Communication 
between the two is a necessary element of 
these procedures. 

CONCLUSIONS 

Excluding the basic flight training 
period, most training in flying is conducted 
without the immediate presence of an 
instructor. This statement is made on the 
premise that flying to maintain proficiency 
is essentially training. Throughout his 
career a pilot receives instrument checks, 
annual proficiency checks, upgrade training, 
and refresher training*' during all of which 



ar> instructor .Or supervisor is very much in 
evidence; but in most of his flying he is 
expected to train himself— to assimilate 
guidance and Instructions on the ground and 
to apply them in the aircraft. Especially 
in practicing takeoffs and landings, in 
instrument flying, and in weapon delivery, 
he learns primarily by self-correction and 
repetition. In brief, he is training by 
acquiring experience. 

It follows that a pilot can learn in 
a simulator also by these methods. -What 
makes self-training effective in an aircraft 
is the pilot's conscientiousness and pro- 
fessionalism; the same characteristics in the 
student can make self-training effective in a 
simulator. There is a difference in the two 
situations, of course. The penalty for lack 
of motivation or a lapse in concentration is 
sometimes fatally severe in an aircraft; in 
a simulator there is no penalty other than 
wasted time and a relatively smalt amount 
of money. 

A controversial aspect of sel f-trarni-ng 
is the effect of the loss of realism caused 
by the student's frequent interacting with 
the simulator. From time to time during a 
self-training exercise, the student will 
take such actions ai reinitializing the 
simulator, entering and clearing malfunc- 
tions, storing flight conditions and 
resetting to the stored conditions, and 
adding fuel and armament. In addition he 
raay display emergency procedures via the CRT 
system and review them before entering mal- 
functions, freeze the trainer and study the 
pertinent approach plates, and., interrupt 
the flight and call up a demonstration. All 
of -these actions would be foreign to his 
procedures as a pilot. The question is, 
would they reduce the transfer of training 
expected from the simulated control re- 
sponses, instrument readings, and visual 
scenes that he is also perceiving? 

It is believed that the answer to that 
question is negative. Again, professionalism 
and conscientiousness will enable the student 
to bridge the gap between managing his own 
training and receiving effective^ training 
experiences. One ingredient is necessary: 
the management task must not become too 
engrossing. 

A related concern is whether the design 
of the student station--and the associated 
software—is sufficiently simple. It would 
be possible to "overwhelm the student by 
attempting to provide all the capabilities 
of an instructor station. Designers will 
have to keep in mind the three required 
characteristics for student station controls 
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cited previously—simplicity, accessibility, 
and inconspicuousness— awj t ihe-tfecessity to 
be conservative in providing instructional 
features. The need for disciplined and 
responsive human factors engineering will be 
possibly greater than for any other area of 
development of a simulator. 

In conclusion, it is recommended that 
specifications for flight simulators (cockpit 
procedures trainers, operational flight 
trainers, weapon system trainers, team 
tactics trainers, and similar training 
devices) include provisions for a student 
sel f- training capability, in addition to a 
conventional instructor station. The users 
will gain tremendous flexibility in simulator 
utilization. In the final analysis, they 
will be able to schedule simulators like 
aircraft— simply, 'some flights can be solo 
and some dual . ~A 
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Fl ight Simula tion has been extens ive ly 
used in both military and cairrercial aviation 
training for the past several decades. Every 
year the ccnplexity and versatility of flight 
simulators grew. However, the full training 
potential of these devices is not often 
realized. There are, of course, a variety of 
reasons why this situation persists. Lack of 
transfer of training due to procedural or 
configurational similarities between the 
simulator and parent aircraft, lack of 
adequate out-tbe~windcw visual displays, and 
the carpleteness of the mission relevant 
aspects of the simulations are but a few. 
However, the point of this paper is that the 
training potential of flight simulators could 
be enhanced by inproving the knowledge and 
understanding of the operating principles of 
the devices , by understanding instructional 
arrangements and by utilizing instructional 
skills. 

Lack of realization of the full training 
. potential of many simulators varies a great 
deal frcm one simulator to another. This 
variance is created by the ccnplexity of the 
mission being simulated, the age and sophis- 
tication of the simulator, the instructor/ 
operator station design, and the availability 
of instructional aids. 

In this paper, a solution to the problem 
is explored— the development of generic 
instructor trainings packages for flight 
simulators. In the following sections of the 
paper , the several general topical areas 
relating to generic instructor training are 
presented. These include the advantages of 
simulator training, the principles of 
effective instruction related to simulator 
instructional features, and possible imple- 
mentation and delivery strategies. 

Advantages of Simulator Training 

Ground- based flight simulators play an 
important role in the overall training scheme 
of most civilian and nearly all military 
aviation training programs. The advantages 
of simulator training are clear. It offers a 
means to provide cost effective and instruc- 
tional ly effective training in a safe environ- 
ment. Cost savings are realized in both the 
obvious and some not so obvious training 
parameters. Obviously, fuel and maintenance 
costs associated with actual flight are 
conserved. Not so obviously, tremendous 
savings in time are realized in simulator 
training. These savings are observed in both 
student and instructor time requirements. 
Students get more relevant training per hour 
of "flight" because they can concentrate on 



the goals of a particular mission while 
-instructors may supervise much more relevant 
training per hour of their comnitirent. 
Simulators also allow a safe training 
environment. In the area of safety, the 
advantages are clear and obvious — simulators 
allcw practice in flight activities where 
the oily negative consequences are learning 
with sane embarrassment and where realistic 
responses to emergency or carpound emergency 
situations may be practiced. Last, but from 
the point of view of this paper, by no means 
least, simulators provide a necessary 
hands-on instructional format for flight 
operations. With simulators as a part of 
the overall flight training regimen , the 
opportunity is .presented to allcw hands-on 
and associated cognitive learning experi- 
ences of the proper training density in 
proximity to the preceding academic events 
and to the subsequent inflight events. 
These simulator events are particularly 
valuable because they provide practice with 
appropriate feedback for the learning 
objectives of interest , as well as allcw 
repeated exposures to particularly difficult 
and important segments of a mission. 

Needless to say, a component of any 
instructor training course related to flight 
simulation should be a thorough discussion 
of these and other advantages of simulation. 

Instructional Principles and Instructional 
Features 

A generic simulator instructor training 
program should amplify a few basic prin- 
ciples of effective instruction particularly 
as they relate to the usual set of instruc- 
tional features found on » most in-service 
flight simulators. The following is a list 
of such principles with their relationship 
to some commonly available instructional 
features. 

1. Student Preparation Leads to 
Ef f ective Learning . It is a well estab- 
lished, even intuitively obvious, fact that 
the better prepared the student is when he 
comes to a simulator exercise, the more he 
will profit from it. A generic simulator 
instructor training program should be 
designed to emphasize this point as well as 
to key the instructor to use some commonly 
available student preparation aids. These 
aids would 'include the formal , simulator 
exercise guides and prompts/briefing aids. 
Formal simulator exercise guides are univer- 
sally available in all Navy training 
squadrons. Some are squadron developed 
while others are the result of contractor 
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supported ISD development activities. The 
generic training should emphasize that 
through the use of these guides both the 
instructor and the student will be prepared 
for the content and flew of the activities 
in the exercise. Also, depending on the 
sophistication of the , documents, the student 
may have an excellent feeling for the level 
of proficiency he is expected to demon- 
strate. At least one simulator program, the 
F-14 OFT program at NAS Miramar, has been 
supported by a very sophisticated device) the 
Instructor Support System, which a 1 lews CRT 
displayed briefing aids. Aids of this type 
could beccrre part of a device-resident 
generic instructor course or any course in 
support of a new I OS design that is suffi- 
ciently complex to contain it. 

2. Accurate and Timely Feedback Leads to 
Inprovements in Performance . Another well 
known principle of effective instruction is 
that accurate and timely feedback' leads to 
irrpr overrents in performance. A properly 
designed simulator instructor training course 
would enphasize this point and relate it to 
the several instructional features which are 
coTiTDnly available on most simulators that 
allow accurate and timely feedback. These 
instructional features include freeze record/ 
replay, automated performance measures, 
graphics displays and hardcopy printouts. 
The freeze function allows immediate feedback 
for the development of fine motor control as 
well as directional control. It is 
particularly useful because it can be 
enployed in , both real time or during a 
performance replay session. Record/replay 
has an obvious application and, in fact, 
provides the basis of many of the other 
instructional features. Automated perform- 
ance rreasOres provide an excellent oppor- 
tunity for lessening the workload of the 
instructor while providing a more accurate 
and complete record of performance than even 
the irost diligent instructor is capable of 
providing. However, while the value of 
automated performance measures should be 
emphasized in a generic simulator instructor 
training program, it must be noted that the 
value of this instructional feature depends 
on the amount of instructor inputs to the 
variables, the weights of the measures and 
the accuracy of the computations. While 
automated performance measurement systems are 
not generally available for most current 
simulators, it is clear that they will be 
available in future systems or in outboard 
support systems. Graphics displays, parti- 
cularly those which display approach, weapons 
delivery, and pattern information, are an 
excellent ireans to provide post-session 
performance feedback. These displays are 
fairly canron on many current simulators, 
although sore are fairly unsophisticated. 
However, future simulator instructor stations 
and support systems for current simulators 
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will include these displays, A clear em- 
phasis should be placed on the value and use 
of these displays in a generic instructor 
course. Hardcopy performance printouts are 
almost universally available on all cenplex 
simulators . However , they are almost 
universally unused as an instructional 
feature. A generic instructor training 
course would enphasize the potential value 
of these printouts as we 1 1 as provide 
suggestions about how they could be sum- 
marized to be a more useful resource. Here 
again, the value and acceptance of these 
printouts depends on the legitimacy of the 
measures expressed. 

3. Practice Improves Performance . 
Another well established principle of 
effective instruction is that practice, 
particularly practice combined with accurate 
and timely feedback, urproves performance. 
As a ratter of fact, it has been conclu- 
sively demonstrated that practice beyond the 
point of proficiency is effective in consol- 
idating learning and making it less likely 
that deterioration in future performance 1 
will occur. Opportunities for practice afe 
provided by both supporting materials and 
instructional features. That is, wi thin the 
supporting materials designed for specific 
training goals, the training objective have 
been arranged to provide practice that is 
related to the difficulty, criticality, and 
frequency of the psychomotor response in 
question. From the point of view of in- 
structional features, the reset function is 
a key feature that allcws repeated practice 
fx>r difficult or problem maneuvers. Reset 
is a carmonly available feature which allcws 
the instructor to reinitiate a training 
routine repeatedly with constant or variable 
flight conditions until satisfactory per- 
formance is observed. Malfunction insertion 
is another- practice-relevant instruction* 1 
feature. With this cartronly ^ available 
feature, the instructor 'can manually or 
automatically insert aircraft malfunctions 
at any stage in a mssion. ^These malfunc- 
tions may be presented once, - * or repeatedly, 
at the instructor's option. This feature 
allcws excellent practice in dealing with 
emergency situations, * 

A generic simulator instructor training 
program would enphasize the importance or 
use of these practice-relevant features. 
The course would include clear direction 
relating to the use and adherence to the' 
supporting materials- Several * problem 
analytic studies have shewn that departure 
from the training routines specified in 
supporting ' materials is a problem in many 
Navy training squadrons. The course would 
also include direction relating to the* 
proper use of the reset and malfunction 
insertion features. This direction should 
enphasize that resetting is an excellent and 




timely method to provide repeated practice 
for problem maneuvers. Ehphasis would be 
placed on the proper use of the malfunction 
insertion features. Here the theme would 
center on adherence to required malfunction 
insertions specified on supporting materials 
and, the proper use of random malfunction 
insertion. The generic course would make it 
clear that intentionally overloading the 
student with coipound emergencies is unreal-? 
istic and may lead to confusion and 
restricted skill advancement in the areas of 
primary ( interest. 

4. Guidance and* Motivation Improves 
Performance . A final generally accepted 
principle of effective instruction related 
to simulator training is that proper guid- 
1 ance and motivation improves performance. 
Surely, one of th^rpst important jcbs of 
the simulator m«rtctor is to provide 
proper guidance Irfe direction to the 
student. The generic course would emphasize 
that this guidance mast be organized, 
mission relevant , and provided in an in- 
structional mode. That is, it must be 
provided in a cooperative setting charac- 
terized by joint achievement student/ 
instructor goals. Several instructional 
features relate to the organization of 
student guidance in the simulator setting. 
These include use of the trainer supporting 
materials and use of the prorrpts/briefing 
aids. These features provide a vehicle for 
guidance. Hcwever, the key vehicle for 
guidance is a sensitive student/ instructor 
relationship. A generic course should 
enphasize that point. Instructor provided 
motivation is another key feature in the 
student/ instructor relationship. A generic 
course should enphasize the need for 
motivational comments, rewards, and open 
carnunication. In addition, the course 
should enphasize the use of some commonly 
available instructional features which may 
aid student motivation. These would include 
'the demons tirat ion feature, the freeze 
feature and all the features that provide 
feedback about performance . A generic 
course should enphasize that demonstrations 
can be motivational ly effective if they are 
not overused or if they are .used as a 
feedback mechanism to provide direction on 
the proper procedures. The freeze feature 
can also have a profound motivational effect 
when it is used to intercept poor 
performance or to indicate a " terminal 11 
condition such as striking the ground or 
being hit by a weapon . The course would 
point out clearly that the instructor is 
responsible for providing motivation through 
the thoughtful use of all of the performance 
feedback features. These would include 
automated performance measures, record/ 
replay, graphics displays, and hardcopy 
printout. The enphasis here should be on 
the proper and sensitive use of these data 
Sources to inprove performance. 



Implementation of Generic Training 

There are several potential advantages 
for a generic simulator training program. 
The program would undoubtedly improve the 
effectiveness of the instructor cadre for 
every simulator where it might be imple- 
mented. This ixrproved effectiveness should 
lead to a more professional approach to the 
overall use of simulator trailing, or a part 
of flight training in general, and to poten- 
tial improvement in student performance. In 
addition, the t course should lead to improve- 
ments in student management from both the 
interpersonal and administrative points-of- 
view. Interpersonal improvements could 
result from the course content in the areas 
of guidance, motivation, and performance 
feedback. Administrative irrprovements could 
result from the generally improved organi- 
zation of simulator training and from the 
use of autarcSfed management aids that4night 
be part of a device-resident training course. 

A disadvantage of the proposed concept 
is that each instructor training course 
associated with a particular simulator would 
have to be tailored to reflect the particular 
instructional features and the nature of the 
supporting materials of that device. This 
requirement admittedly diminishes the generic 
value of the concept but, hopefully, only a 
snail amount of tailoring would be required 
to address most simulators. The training 
course could be implemented in two distinctly 
different formats — device-resident or device- 
independent. In the device- resident format, 
the course could be contained. in the software 
of the simulator conputer and presented on a 
CRT associated with the IOS, or a supporting 
system. With this format, the course would be 
continuously available, obviously convenient, 
and might even be made contingent for in- 
structor utilization of the device. In the 
device- independent format, the course could 
be presented in a stand alone media, such as 
a workbook or slide/ tape, and could be pre- 
sented in an individualized study atmosphere 
in a learning center^ or training support 
center. This format would allcw instructor 
access in a self-paced, individualized 
arrangement which is consistent with the 
manning demands of- most training squadrons. 
In either forthat, the course would demand a 
nun imam of intrusion into training squadron 
activities during its development while 
providing a maximum of benefit for inst»*etQr 
training. \ 
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INTRODUCTION 



The objective of the research reported 
here is to evaluate the feasibility and useful- 
ness of applying two emerging technologies to 
naval training. These technologies are 
artifical intelligence and voice recognition P 
and synthesis. 

Artifical intelligence (AI) is a body of 
concepts and techniques which have been devel- 
oped to permit computers/machines to do some 
of the complexr-cognitive activities that have 
been regarded traditionally as the unique 
providence of huuan baings, in some instances 
exceptionally talented human beings. Rep- 
resentative cognitive activities are problem 
formulation, searching for problem solutions, 
diagnosis, and decisionmaking. These 
activities require representation of knowledge 
and inferential reasoning about that knowledge. 

If these concepts can be combined with - 
voice recognition and synthesis, then an inter- 
active, oral dialogue can take place; the 
verbal exchanges can be flexible, meaningful, 
and appropriate to the context of the on-going 
events. Applied to training and education, 
computer-aided instruction moves a major step 
' in the direction of a realistic journeyman- 
apprentice relationship. The journeyman can 
be designed to be not only a technical expert 
but an expert tutor as well . 

The organization of this paper consists 
of four parts : 

Review and Evaluation of the Component 
Technologies and Principal Concepts in 
Artifical Intelligence 

Review of Naval Training Needs Suitable 
for a Prototype Course Using AI and Voice 
Recognition/Synthesis 

Recommended Guidel ines 'for Instructor 
Support 

Discussion of Implications and Issues 

This research has been funded through two 
sources. One source is an on-going, multi- 
year project under Honeywell's program of 
Independent Research and Development. The 
second source is a contract with the Naval 
Training Equipment Center (NTEC 81-C-0093-13) 
entitled "Use of Voice Technology a the 
Instructor's Assistant." 



, It is assumed that the demographic pre- 
dictions of unavailability of future military 
manpower are accurate. Therefore, it will be 
necessary tp develop more efficient training 
systems and instructional methodology that are 
more effective in amount of competence produced 
per hour of instruction. The proposed approach 
is feasible and will provide a training capa- 
bility to meet the need. The cost, versus 
benefit is more difficult to assess, however. 
We can assume that software costs wilT continue 
to accelerate and that all technology is expen- 
sive to develop. Intelligent systems of this 
type are software intensive. Further, the 
technological development needed is undeter- 
mined but certainly not trivial. 

Artifical intelligence and voice tech- 
nology may no longer be embryonic but they are 
far from mature for most applications. 

REVIE 1 .,' AND EVALUA TION OF THE COMPONENT 

itCHNOLOGIEr«ron>R I N CI PAL CONC EPT S IN 

ainincAL intelligent 

The needs of the Navy in instructional 
capability were examined briefly for the 
purpose of guiding the survey of technology. 
The results of that survey were that the Navy 
needs instructor support, friendly and for- 
giving interfaces between the training devices 
and both the instructor and student, and 
automated performance measurement. The 
functions and tasks for which instructor 
support is needed ere: ■ • * 

Subject Matter Expertise 

Performance Evaluation ♦ 

Problem Selection 

Delivering Feedback 

Progress Management 

Documentation 

Role Playing 

The review of A I technology was organized 
into the following topical categories: 

Knowledge Representation 
Learning Theory 
The Nature of; Expertise 
Intelligence Computer Aided Instruction 
Software Architecture for Artifiea-1 
Intelligence 

Natural Language Understanding 
Interactive Speech 



The n^jor conclusions will be summarized 
briefly. 
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The area of Knowledge Representation is 
known as Domain Knowledge. There are two 
relevant domains for a training system: the 
knowledge content of the technical specialty 
being taught and the instructional knowledge 
of methods, strategies, and procedures. They 
present no unresolvable problems for knowledge 
representation. The Content of the domains 
w» 1 1 become manifest as the training materi- 
als are developed. However, not all of the 
necessary knowledge has been explicitly 
articulated or well-defined. 



An important area of Instructional 
knowledge is the evaluation of m student perfor- 
mance. The level of effectiveness of a train- 
ing system is dependent on the availability 
of two kinds of evaluative data: 

Assessment of the leve-1 of -performance 
in terms of level of competence and one's 
standing in the subject matter. 

Diagnosis of errors committed in terms of 
specific deficiencies in knowledge or 
skill and necessary corrective action. 



Existing techniques 
resentation are adequa^ 
intelligent, interact rve 
support- an instructor, 
si st of representing thi 
plates, frames, and scri 
ships among them in cont 
tion rules. These rules 
to knowledge derived int 



f knowledge rep- 
r for constructing an 
training device to 
These techniques con- 
ngs and events^ in tem- 
pts and the relation- 
rol rules and produc- 
also govern additions 
ernally by inference. 



The contribution of Learning Theory is 
primarily in providing a structure and method- 
ology for preparing training materials. One's 
first expectation might be that learning 
theory would provide models of the learning 
process; however, it performs that -function 
only indirectly. Learning theory will provide 
a generic, transportable framework and methods 
for developing courseware and rules and tech- 
niques for performing task analysis. These 
methods and techniques entail specification of 
stimuli, acceptable and unacceptable responses, 
feedback, sequencing, branching, display for- 
mats and so on; they are in turn a function of 
the domain content, training objectives, 
instructional strategy, media, rules for per- 
formance assessment, and specific instructional 
techniques/ All of this is dependent on the 
learning process. However, that knowledge 
must be encoded into the rules forinstructional 
expertise and it will be transparent in the 
application. 

Our interest in the nature of expertise 
is how the knowledge of competent practi- 
tioners should be used in the training 
system. Examination of the representation of 
that knowledge revealed that the form of rep- 
resentation varies with domain. Domains 



differ in structure encompassing the gamut 
from production rules to hierarchical organ- 
ization; multiple conceptualizations for the 
same -phenomena are available in some domains; 
multiple levels of abstraction are character- 
istic of some domains. 

Problem solving methods also varied across 
domains. Different methods are available for 
different tasks. A given problem solving 
method is also adapted to the requirements and 
constraint^ of specific task conditions. 

Therefore, it seems unlikely that exper- 
tise can be captured in a few, simple, generic 
algori chins or procedures that are general iz- 
able across knowledge domains or even within 
a domain. The interaction between the nature 
of expertise and knowledge representation is 
perhaps the most critical element in develop- 
ing an intelligent training sysrtem. Knowledge 
structures and control rules that can represent 
the complex contingencies will have to be 
developed. The subject matter to be learned 
at novice levels will probably be relatively 
simple and without much of the troublesome 
complexity. This problem must be faced, 
though, when the' training moves into more 
sophisticated or intermediate level topics. 

One perspective on the purpose for devel- 
ooinq these intelligent systems is to 
shorten the transition time from novice to 
expert. We want to produce higher levels 
of competence in shorter periods if instruction 
and with lower support costs. Ideally, this 
tran$ubstantation will be accomplished on 
trainees who are naive with respect to the 
technical area and of no more than average 
intelligence (general intellectual ability). 
An instructional system that will accomplish 
this feat will heed a model of the sequences 
of developmental stages from novice to expert,, 
states within those stages, and the pedagogical 
techniques for transformation between states 
or stages. At the present state of our tech- 
nology such a model must be derived within the 
context of a specific domain. 

The Software Architecture for Artifical 
Intelligence was viewed as consisting of two 
parts: the hardware necessary to execute the 
program and the software. The hardware was 
found to be specific to the training program 
and thus no generalizations can be made. It 
was concluded that LISP is the software system 
that should be used for development and 
operation of such' training systems. Knowledge 
representation techniques gxist in LISP as 
well as an assortment of supporting tools and 
facilities. Further, there is a history of 
experience with this language which provides 
both a body of knowledge and experience AI 
practioners to draw upon. 

Interactive Speech and Natural Language 
Understanding present no obstacle in principle 
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although deriving and organizing the detailed 
dialogue requirements can be a challenging 
management task. Spoken messages can be 
synthesized at the current state-of-the-art. 
Generation of the messages in the context of 
on-going events and conditions and in an 
adequate approximation to real time i.s uncer- 
tain and must be determined in the specific 
applications. 

Current and near future voice recognition 
systems can recognize utterances of only a few 
words in length; recognition ^end understanding 
of unconstrained, connected speech is not 
feasible and may not be practical for some 
time. This character of voice technology 
imposes a requirement that the natural lan- 
guage dialogue consist of short statements. 
Short statements may be characteristic of 
speeeh i^task-griented _$ituatipns and thus 
this constraint may not be a practical 
1 imitation. 

REVIEW OF NAVAL TRAINING NEEDS SUITABLE FOR 
A PROTOTYPE COURSE " 

Four training areas were reviewed for 
suitabil ity. They are: 

Electronic Maintenance 

Air Intercept Control 

i^aval Flight Officer Course 

Ant i -Submarine Warfare Team Training 

In addition, training equipment in team 
training and air traffic control was analyzed. 
The team trainers analyzed were TACDEW, 
14A2 ASW, and 14A12; the air traffic control 
trainers were PARTS (McCauley and Semple, 
1980) and the AIC Trainer (Hal ley, King, and 
Regelson, 1978). 

Tbe criteria used to evaluate the train- 
ers were cost-benefit, availability/shortfall 
of instructors, appropriateness of content 
to AI and voice, impact on readiness, and 
stability of the subjedf matter. The cost- 
benefit, criterion was broken down into three 
parts. First, the developmental cost of the 
prototype v/as treated in terms of the 
availability of appropriate informatipn, 
data, and on-going training. Second, operat- 
ing cost v/as treated as cost of instructors 
as the major factor differentiating among 
courses. The third part was a payback com- 
ponent in terms of readiness of trained per- 
sonnel and reduction in personnel replacement. 

Appropriateness of the subject matter for 
the use of artifical intelligence and inter- 
active voice is an important methodological 
consideration. The prototype application must 
be able to reflect an advantage from utilizing 
the new technologies. Artifical intelligence 
will be beneficial in domains that have a 
significant cognitive component;* voice tech- 
nology is useful in areas of high workload 



which can be reduced by using hearing and « 
speech as media of interaction # rather than 
vision and manual responses. 

ASW team training was chosen out of the 
four training areas as the best candidate for 
a prototype application. Electronic main- 
tenance was rejected because it does not have 
<a significant voice potential and develop- 
mental costs could be high. Maintenance- 
skills are poorly understood and significant 
costs and delay could be incurred in getting 
necessary information. Air intercept control 
and the naval flight officers course were 
rejected because the subject matter is not 
stable. 

RECOMMENDED GUIDELINES FOR INSTRUCTOR SUPPORT 

.Analysis of the instruction yielded 
needs for jt he -f ol lowing modules for , 
instructor support: 

Record Keeping^ 
Scenario Set Up 
Subject Matter Expert 
* Subject Matter Tutor 
Speech Generation 
Speech Recognition 
Ability to Underhand Errors 
Omniscient Instructor 

The major components of ari Automated 
Training System are: fr 

Instructor Work Station 
Student Work Station 
Domain Expert 

Instructional Expert <, . 

Performance Measurement 

The Instructor Wotfk Station was examined 
'in more detail to identify the capability it 
should have. The Instructor Work Station 
should have the fallowing functions: 

r 1 

Explanation of Decisions and Actions 
Record Keeping and Scheduling 
Scenario Design and Debug" * 
Communications Control m : . 

• Problem Status 4 * * &}' 

An important capability of the system is 
to 'recognize and cope with errors committed* 
by the trainee. This. capabi 1 ity entails 
several requirements in knowledge and use of 
the knowledge to make inferences about the 
trainee's behavior. The system must know the 
constraints reflected in the rules for correct 
behavior. Since the application is ASW team 
training, the system must also know how the 
trainee deals with other team members. Each 
player must |<now the roles of the other 
players at a minimal level of their nominal, 
expected behaviors in«relation to one's own 
role. 



Therefore, the system must know* hot only 
the role structure of the team but also each' 
trainee's, beliefs about knowledge and com- 
petence of other players. It must be able to 
follow a trainee's reasoning about the knowl- 
edge of other players and know whether he. 
attributes erroneous or unexpected behavfor 
from another player to a lack of knowledge, 
misinformation, or faulty, missing, or in- 
appropriate procedures. The inferences the 
trainee makes and how, he responds to the other 
player's behavior are relevant; he may fill 
in missing information, correct the other 
player, request verification, or adjust his 
own behavior to compensate for the perceived - 
deficiencies in the other player's behavior. 

These'capabi 1 ities have been clustered 
into four sets which represent increasing, 
levels capability ar^gaelated to three stages 
in developing an inrcmgent, automated in- 
structor or instructor's, assistant, based on 
Crowe et al., (1981). We have modified the 
model of Crowe et al . by adding a fourth 
stage consisting of the ability to provide 
feedback and performance evaluation to. team 
behaviors. The' cross-correlation of these 
classifications is depicted in Table 1. 

The correspondence between the major 
components of an Automated Training System 
and the Al technological areas on which they 
depend is summarized in Table 2. * • 

DISCUSSION^ OF IMPLICATIONS AND ISSUES 

Qmt intent in this 'Section is to be 
explicit about some issues and concerns that . 
were recurrent during our analyses and set 
aside for the purpose of attaining the 
immediate goal of developing some guidelines 
for how to proceed in developing an simulated, 
intelligent, talking instructor's assistant. 
They should at least be stated openly because 
they may represent potential problems or 
obstacles to developing, fielding, and 
effectiveness of this type of system. The 
intrepid innovator can then beware end 
take appropriate cautionary or preemptive 
actions. 

These issues and impl icatipns are: 

Impact on the Instructor 

Reduction of Personal .Contact betweerr 

the Student and Instructor 
The Achievable Level of Naturalness 
Effect of Complexity on Friendliness 

and. Support 
importance of Message Generation Versus 
„ Voice Synthesis in the User-System 

Interface 
Compressibility of the- "Novice-To- 

Expert" Curve * 



There are several possible effects on the 
instructor and his- role .which can have a neg- 
ative impact on system effectiveness. The 
content of instructor training will necessarily 
be changed and perhaps increased in length and 
difficulty. Our intuition tells us that the 
instructor will need a better understanding of 
the instructional process than he presently 
has. At the same time his automated assistant 
wiH permit him to handle more trainees and 
attain 1 higher levels of proficiency; his 
productivity should undergo a marked enhance- 
ment. However, without an instructional ly 
sophisticated supervisor the automated in- 
structor's assistance can run amok 1 ike the « 
sorcerer's apprentice. 

The instructor's morale, involvement, and 
sense of using his own talents wiJl be affect- 
ed. These factors will be enhanced if the in- 
structor perceives his automated/assistant as 
a^ool which helps him indoinq/his job, as 
something to which he delegate! tasks and 
which' increases his effectiveness and permits 
him to do a better job. If irrcontrast he 
sees the "iron monster" as something which he 
must labor to support and dominates his 
activities, then his morale, motivation, and 
involvement will decline. 

Reduction of personal contact Between the 
instructor and trainee can compromise the 
effectiveness of social reinforces and 
attenuate an important source' of feedback and 
diagnostic information which every good in- 
structor learns to use. It will also minimize 
the opportunity to use a mentor relationship 
to model behaviors and facilitate the instruc- 
tional process. The voi*ce of the simulated 
assistant may take on a personality and a 
reality as an individual and thus be able to 
satxsfy some of these social functions. 
Realizing them may be a function of the 
creativity in preparation of the dialogues, 
the types interactions embedded in them, and 
the naturalness of the interaction as per- 
ceived by the trainee. The personal ity of the 
automated assistant may be another domain of 
expertise needed to develop and deploy these 
systems. • \ 

•The achievable level of naturalness is 
unknown at this time as is the required level 
of naturalness. Naturalness and friendliness 
are concepts that are not well definech and 
have a large subjective, intuitive component 
as the terms are used. Specifications are 
difficult to state arid satisfy under such 
conditions. These terms need to.be differ- 
entiated and made more^ precis^. 

v The impact of naturalness and the com- 
plexity of the interactions on system sbpport " 
and maintenance is a very serious issue. Tl?e. 
transparency of natural, friendly, simpVe 
dialogues is achieved by putting the com- 
plexity and processing behind the interfacing 
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console and into 'the software. .'Experience 
tells us that' maintenance and support^in- 
crease as software and, hardware become more 
-complex; Murphy's Law is imperative. 

Voice synthesis with some degree of 
natural sounding speech is relatively easy to 
.achieve in the current state of the art. 
However, the appropriateness of the message 
in the context of on-going events and the - 
speech community of the listener are more 
difficult to accomplish. These features are 
the problem of message generation. It is 
dependent on a/lot of information processing 
iavolving knowledge representation, inference, 
and domain knowledge. This area^s critical 
to the -effective use of voice technology. 

Compressibility of the curve for trans- 
foxing a -novice into some level ol^fexpert 
is an ultimate instructional issue. There 
are many differences between novice and 
expert in amount of knowledge, its organ- 
ization, techniques for formulating and 
solving problems, and tfie use of heuristics 
derived from experience and the accumulate 
lore of the discipline. A subset of these 
things must be chosen in terms of their 
utility on the job for a given level of 
proficiency, the ease of learning- them, and * 
the progression of states of knowledge and 
still in the growth cJf ,prof ici ency . There 
is a multitude of empirical questions in- 
herent in this issue; they must jo unscramb- 
led afid addressed if we are to have an effect- 
ive technology 'for 'autOMation x>f instruct- 
ional systems. . 

C ONCLUDING REMARK S 

Application of artifical intelligeoce 
"and voice technology to training systems in 
the f(frm of an automated instructor's 
assistant is feasible and can provide sig- 
nificant uicrefases in the cost-effectiveness 
of training. Artifical intelligence provides 
several concepts and techniques which are 
useful in^ structuring and implementing the 
design and use of such a system. This paper 
is a brief "summary of af> on-going investi- 
gation of the feasibility, util-i'ty, and 
approach to developing an automated 
instructor's assistant for. use in training 
Navy personnel.. ASW team training was chosen 
as the vehicle for a prototype application 
, of the automated instructor's assistant. 
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